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, 1 . INTRODUCTION 

. J . ' 

This  semiannual,  technical  report  covers  research  carried  out  by 
the  Advanced  Teleprocessing  Systems  group  at  UCLA.. under  ARPA  contract 
, , DAHC  15-73-C-0368  during  the  period  of  January  1,  1976  to  June  30,  1976. 

j Considerable  progress  was  made  on  all  four  contracted  tasks,  namely 
.ground  radio  packet  switching,  satellite  studies,  resource  sharing,  and 
security.  . In  the  following  paragraphs  we  describe  that  progress  and 
PQint  to  the  list  of  references  which  represents  the  published  work 
resulting  from  this  supported  research. 

Following  this  summary  is'  a list  of  -publications  produced  as  a 
» result  of  the  recent  research  on  this  contract  covering  the  six  months 
being  reported  on.  As  usual  we  devote  the  main  body  of  this  report 
to  the  detailed  presentation  of  one  aspect  of  this  overall  research, 

< and  simply  mention  the  other  areas  briefly  in  this  summary. 

\ ' 

The  research  which  is  detailed  in  the  main  body  of  this  report 
refers  to  the  design  of  very  large  computer  networks.  The  key  problem 
here  is  that  standard  design  and  routing  -techniques  fail  when  networks 
' ' ^.become  very  large  due  to  the  cost  of  design  (which  grows  exponentially 
.with  the  number  of  nodes  in  the  network;  this  exponent  varying  from 
. three  to  six  depending  upon  the  particular  design  technique  used)  and 
, to  the  length  of  the  routing  tables  (which,  with  the  usual  adaptive 
routing  procedures  require  table  lengths  equal  to  the  number  of  nodes 
in  the  network).  The  research  repotted  upon  here,  which  represents 
/*  .'the  Ph.D.  dissertation  of  Farouk  Kamoun,  (Chairman,  Leonard  Kleinrock) 

* solves  both  of  these  problems  using  the  familiar  technique  of  hierarchial 
design  and  routing.  However  the  resulting  network  need  not  be  a hier- 
archical network  in  the  usual  sense,  but  in  fact  may  simply  impose  a 

- hierarchical  structure  on  an  otherwise  distributed  network.  In  this 

• * fashion  the  design  costs  grow  exponentially  with  m,  where  m represents 

the  size  of  basic  partition  into  which  the  network  is  decomposed;  this 
^ offers  enormous  savings  in  design.  The  routing  problem  is  also  solved 
using  area  routing  (clusters,  superclusters,  etc.);  here  we  are  able 
to  prove  first  that  in  the  limit  of  very  large  networks,  the  relative 
'table  length  shrinks  to  zero  whereas  the  relative  path  length  grows 
not  at  all  as  compared  to  a network  without  a hierarchical  routing 
structure.  Not  only  do  table  lengths  bother  us  in  terms  of  storage 
used  in  IMPS,  but  they  bury  us  when  it  comes  to  updating  these  tables 
both  in  the  form  of  line  overhead  for  the  routing  of  their  messages  and 
in  the  enormous  IMP  processing  overhead  in  order  to  reduce  the  tables  as 
, those  updates  are  received.  Indeed  one  easily  shows  that  nonhierarchical 


i 
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routing  techniques  cannot  be  supported  in  large  networks;  hierarchical 
routing  solves  this  problem  nicely  and  we  are  able  to  show  again  in 
limit  of  large  nets  that  relative  table  lengths  go  to  zero  whereas 
no  loss  in  throughput  need  be  suffered.  Furthermore  it  is  surprising 
that  networks  on  the  order  of  one  hundred  or  two  hundred  nodes  begin 
to  benefit  from  hierarchically  routed  structures!  Further  details 
can  be  found  in  the  abstract  and  dissertation  which  we  have  reproduced 
in  the  main  body  of  this  report. 

Our  efforts  in  Task  I,  Packet  Radio  Studies,  have  met  with  consider- 
able success  in  this  period.  The  Ph.D.  dissertation  of  Michel  Scholl 
is  near  completion  and  a number  of  new  protocols  have  been  developed 
which  compare  very  nicely  with  all  known  protocols  and  begin  to 
approach  that  of  perfect  scheduling.  This  material  will  be  reported 
upon  in  a future  Semiannual  Technical  Report.  Further  progress  in  the 
carrier  sense,  multiple  access  schemes  including  the  hidden  terminal 
problem  and  the  split  reservation  multiple  access  technique  has  been 
made.  The  experimental  packet  radio  system  under  the  supervision  of 
Stanford  Research  Institute  has  progressed  in  this  last  period  and 
considerable  details  of  the  measurement  code  and  measurement  functions 
have  been  defined  and  designed  for  use  in  that  experiment. 

Task  II,  Satellite  Experimentation,  has  been  a hot  bed  of  activity. 
We  have  made  numerous  measurements  on  the  experimental  satellite  channel 
and  have  been  able  to  study,  various  multiple  access  techniques  both 
through  these  measurements  and  through  simulation  and  theory  supporting 
these  measurements.  This  effort,  under  the  leadership  of  Linkabit 
Corporation  and  in  conjunction  with  other  groups  across  the  country 
and  in  Europe  have  provided  a wealth  of  material  and  understanding  in 
multiaccess  broadcast  satellite  channels. 


Resource  Allocation  and  Sharing  are  represented  in  Task  III.  The 
work  of  Farouk  Kamoun,  mentioned  above,  falls  in  this  category.  In 
addition  Zipora  Erlich  is  completing  her  Ph.D.  dissertation  on  resource 
allocation  of  finite  capacity  buses  in  a probablistic  environment;  this 
work  extends  some  of  the  queueing  theoretic  models  useful  in  packet 
radio  as  well  as  represents  an  application  of  packet  radio  in  a natural 
environment.  In  our  effort  to  study  the  effect  of  a packet  switching 
network  on  real  time  signals  such  as  packetized  speech,  we  have  observed 
the  effect  of  routing  updates  on  such  a real  time  packet  flow  and  have 
been  able  to  analyze  and  model  these  effects.  All  of  this  work  is 
reported  upon  in  the  various  publications  which  are  listed  in  the  bibliO' 
graphy  following  this  summary. 


During  this  period  significant  progress  was  made  in  several  area  of 
Task  IV.  The  virtual  machine  system  was  fully  converted  to  the  PASCAL 
language  and  extensively  debugged.  In  addition  to  making  the  system  viable 
and  maintainable,  the  conversion  actually  improved  its  performance. 


Improvements  were  also  made  in  the  UCLA  PASCAL  translator  to  support 
the  activity.  Work  continued  at  an  accelerated  pace  on  the  design 
of  the  UCLA  Secure  UNIX  prototype.  A number  of  the  decisions  which 
were  made  were  documented  and  discussed  with  members  of  the  MITRE 
corporation.  Efforts  began  on  the  modification  of  the  UCLA  kernel 
to  support  the  UNIX  system.  In  related  activities,  progress  was 
made  on  the  verification  of  the  UCLA  kernel  source  code.  The  spec- 
ifications for  the  concrete  source  code  were  refined,  and  experience 
was  gained  in  the  use  of  the  ISI  verification  system.  A simplified 
version  of  one  of  the  UCLA  kernel  routines  was  successfully  proven 
using  that  system.  One  of  the  members  of  the  UCLA  research  group 
served  as  chairman  of  a committee  composed  of  noted  language  researchers. 
This  group  is  developing  a new  programming  language  to  incorporate 
recently  gained  knowledge  of  how  to  enhance  the  security  and  verifi- 
ability of  systems  software.  A draft  report  on  the  new  language, 

EUCLID,  has  been  circulated  privately  in  the  research  community,  and 
development  of  the  final  report  is  in  progress. 

In  addition  to  the  work  reported  upon  in  the  following  bibliography 
numberous  internal  reports  and  memoranda  have  been  written  by  our  group. 
Further,  a number  of  additional  papers  have  been  submitted  for  publication 
in  the  professional  literature. 
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ABSTRACT 


^This  research  deals  with  the  specification,  analysis  and 
evaluation  of  some  routing  and  topology  design  procedures  for  large 
store  and  forward  packet  switched  computer  communication  networks.  The 
procedures  studied  are  an  extension  of  present  techniques  and  rely  on  a 
hierarchical  clustering  of  the  network  nodes. 


Hierarchical  adaptive  routing  schemes  are  investigated  in  the 
context  of  large  networks*.  In  particular,  optimal  clustering  structures 
are  determined  so  as  to  minimize  the  length  of  the  routing  tables  used 
by  the  routing  function,  me  effect  of  this  reduction  of  routing  infor- 
mation on  the  message  patJl  length  is  evaluated.  Queueing  models  are 
developed  in  order  to  evaluate  the  delay-throughput  performance  of  the 


hierarchical  routing  policies  and  to  compare  them  with  present  schemes. 
The  models  prove  the'  infeasibility  of  present  routing  procedures  for 
large  networks  ar yd  demonstrate  the  remarkable  efficiency  of  hierarchical 
routing  scheme^ 


Thffse  queueing  models  represent  an  extension  of  Kleinrock's 
model  for'  networks  in  that  they  consider  nodal  storage  requirements  and 


line  overhead  due  to  routing  updates.  Furthermore,  several  buffer 


sharing  schemes  are  proposed  and  analyzed  in  order  to  optimally  utilize 


nodal  storage. 

*jA  hierarchical  design  methodology  is  also  presented  as  a 
solution  for  the  reduction  of  the  computational  cost  involved  in  the 
topological  design  of  large  networks.  A computational  cost  model  is 
developed  in  order  to  determine  that  clustering  structure  which  achieves 
a minimal  cost.  Such  optimal  structures  are  fouid  which  lead  to  very 


significant  savings  in  computation 
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CHAPTER  1 


INTRODUCTION 

One  of  the  main  reasons  for  the  great  interest  in  computer  net- 
works is  the  considerable  economy  that  can  be  achieved  through  resource 
sharing  [ROBE  70].  Among  such  resources  we  include  computer  power  for 
load  sharing,  specialized  hardware,  specialized  software,  data  banks, 
etc. 

Such  networks  are  called  distributed  computer  communication  net- 
works and  they  made  their  first  appearance  with  the  ARPANET  [HEAR  70], 
[KLEI  70] , [FRAN  70] , [CARR  70]  , [ROBE  70] . 

Computer  networks  are  also  emerging  as  very  efficient  means  for 
data  comiiuni cation  between  remote  locations.  The  first  coimnercial  data 
carrier,  TELENET,  is  already  operational. 

At  the  root  of  this  booming  demand  for  conputer  networks  is  the 
®ver  increasing  need  for  computer  and  data  communication  powers.  It  is 
projected  that  by  1980  approximately  a quarter  miT.lion  terminals  will  be 
in  use  in  Europe  [EURO  73],  [PETE  73]  and  as  many  as  four  or  five  million 
in  the  United  States  [KLEI  76]. 

A very  important  component  of  the  network  is  the  communi cation 
subnetwork.  This  includes  the  hardware  and  software  specifically  dedi- 
cated to  the  transfer  of  data  from  node  to  node.  Many  alternative 
co*umication  schemes  can  be  implemented  at  the  subnet  level.  Among 
these  are:  circuit  switching  [PORT  71],  packet  switching  (a  form  of 

store-and-forward  communication)  [KLEI  64],  radio  broadcasting  [ABRA  70], 
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satellite  communication  [LAM  74],  or  any  combination  of  the  above,  etc. 

The  selection  of  the  best  scheme  is  a difficult  problem  and 
depends  very  much  on  the  nature  of  the  traffic  to  be  handled  by  the  net- 
work [CIDSS  72A,  72B],  [MIYA  75].  The  bursty  nature  of  conqputer  traffic, 
as  well  as  the  continuously  decreasing  cost  of  computer  hardware 
[ROBE  74],  very  much  favor  the  packet  switching  as  the  technology  for  us 
to  consider. 

The  basic  concepts  and  the  first  packet  switching  computer  net- 
work were  developed  by  the  United  States  Department  of  Defense  Advanced 
Research  Projects  Agency  (ARPA) . This  network  (ARPANET) , in  operation 
since  1969,  has  been  an  enormously  successful  demonstration  of  the 
packet  switching  technique.  It  has  resulted  in  the  appearance  of  a 
multitude  of  other  networks  throughout  the  world  (the  NPL  network  in 
England,  CYCLADES  in  France,  etc.). 

Present  computer  networks  may  be  characterized  as  small  to 
moderate  sized  networks  (57  nodes  for  the  ARPANET  as  of  December  1975). 
The  predictions  mentioned  above  indicate  that,  in  fact,  large  networks 
of  the  order  of  hundreds  (or  even  possibly  thousands)  of  nodes  are 
imminent. 

In  the  course  of  developing  the  ARPA  network,  a design  method- 
ology has  evolved  which  is  quite  suitable  for  the  efficient  design  of 
small  and  moderate  sized  networks  [FRAN  72],  [GERL  73A].  Unfortunately 
the  cost  of  conducting  the  design  is  prohibitive  if  these  same  tech- 
niques are  extrapolated  to  the  large  network  case.  Indeed,  not  only 
does  the  cost  of  design  grow  exponentially  with  the  network  size,  but 
also  the  cost  of  a straightforward  adaptive  routing  procedure  becomes 
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prohibitive.  Other  design  and  operational  procedures  (techniques)  must 
be  found  which  handle  the  large  network  case  and  such  techniques  form 
the  subject  of  this  dissertation. 


Routing  for  Packet-Switching  Networks 

In  a packet  switching  network,  messages  are  partitioned  into 
segments  called  packets  which  then  are  transmitted  through  the  network 
using  the  store- and- forward  switching.  That  is,  a packet  traveling  from 
source  S to  destination  D is  received  and  "stored"  in  queue  at  any  inter- 
mediate node  K while  awaiting  transmission,  and  is  then  sent  "forward" 
to  node  P,  the  next  node  on  the  route  from  S to  D,  when  channel  (K,P) 
permits. 

The  selection  of  the  next  node  P is  made  by  a well-defined 
decision  rule  referred  to  as  the  routing  policy.  Several  classification 
schemes  have  been  devised  to  characterize  routing  policies  [KLEI  $4], 
[FULT  72],  [GERL  73B] , [MCCO  75],  [MCQU  74].  Generally  speaking,  routing 
policies  may  be  divided  into  two  main  classes:  deterministic  and  adap- 
tive. While  deterministic  routing  is  more  attractive  to  use  at  the 
design  phase,  adaptive  policies  are  essential  for  the  operation  of  the 
network.  A main  objective  of  this  dissertation  is  to  specify  and 
evaluate  adaptive  routing  policies  for  large  networks. 

The  major  goal  of  adaptive  routing  policies  is  to  sense  changes 
in  the  traffic  distribution  and  network  status  and  then  to  route  messages 
such  that  the  conjested  and  damaged  areas  of  the  network  are  avoided. 

As  a consequence,  they  adjust  to  load  fluctuations  and  node  or  branch 
failures.  The  most  commonly  used  adaptive  policies  base  their  decision 
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on  routing  information  (delay  estimate,  excess  capacity  estimate,  hop 
number  estimate)  stored  in  tables  at  each  node  of  the  network.  These 
tables  identify  the  output  line  to  select  for  each  destination  in  the 
network.  They  are  updated  periodically  or  asynchronously,  or  a com- 
bination of  both,  using  routing  information  collected  internally  (at 
each  node)  and/or  provided  from  neighboring  nodes. 

For  large  computer  networks  (on  the  order  of  many  thousands  of 
nodes  or  more) , the  length  of  the  routing  table  (which  directs  the  traf- 
fic through  each  node)  will  grow  lineraly  (one  entry  per  node)  with  the 
number  of  nodes,  and  therefore  the  storage  required  to  contain  this  list 
in  each  node  will  be  extremely  costly.  Also,  as  a direct  consequence  of 
these  large  table  lengths,  the  cost  of  interchanging  routing  information 
among  the  network  nodes  will  also  grow  and  will  represent  a significant 
burden  on  the  communication  lines  themselves.  All  these  considerations 
suggest  that  a reduction  of  routing  table  length  of  some  sort  is  called 


The  main  idea  in  reducing  the  table  length  is  to  provide  at  any 
node's  (say  i)  routing  table,  one  entry  per  destination  node  for  those 
nodes  which  are  close  to  i (in  terms  of  a hop  distance  or  some  nearness 
measure)  and  one  entry  per  set  of  nodes  for  those  nodes  located  further 
away  from  i.  The  size  of  this  set  may  increase  with  the  (average)  dis- 
tance from  i to  the  set  of  nodes.  This  partitioni:  » of  nodes  into  sets 
may  be  realized  through  a hierarchical  clustering  of  the  nodes.  Consider 
a large  distributed  network  and  assume  that  we  can  realize  the  grouping 
of  nodes  into  clusters,  clusters  into  superclusters,  etc.  Indeed,  let  us 
assume  that  there  is  to  be  an  m-level  hierarchy.  Provided  such  a 
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clustering  structure,  a routing  table  at  any  node  (say  i)  will  contain 
an  entry  for  each  node  in  the  same  cluster  as  i,  and  an  entry  per  clus- 
ter in  the  same  supercluster  as  i,  etc.,  thus  achieving  a reduction  of 
routing  information. 

It  is  such  a scheme  that  we  intend  to  define,  analyze,  and  dis- 
cuss. Fultz  [FULT  72],  McQuillan  [MCQU  74]  and  others  proposed  similar 
schemes  but  did  not  provide  any  quantitative  or  experimental  analysis 
as  we  do  here. 


1.2  The  Design  of  Computer  Networks 

In  our  previous  considerations,  we  were  given  a large  distrib- 
uted computer  communication  network  and  the  problem  was  to  devise  an 
appropriate  adaptive  routing  scheme  which  would  operate  efficiently  with 
a fairly  small  amount  of  routing  information.  Now,  going  one  step  back, 
we  are  interested  in  designing  the  topology  of  a large  network  under 
some  cost  and  performance  constraints. 

Several  different  formulations  of  the  design  problem  related  to 
the  communication  subnetwork  can  be  found  in  the  literature  [FRAN  72], 
[GERL  73A].  Generally  they  correspond  to  different  choices  of  perfor- 
mance measures,  design  variables  and  constraints.  Here,  we  select  the 
following  very  general  formulations. 

Given:  Node  locations 

Channel  capacity  options 

Minimize:  Total  communication  cost 
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Over:  Topology 

Channel  capacities 
Routing  policies 
Subject  to:  Delay  constraint 

I 

Reliability  constraint 
Traffic  requirement 

Along  those  formulations,  several  solutions  have  been  proposed  and 
applied  to  ARPA-like  network  designs.  However,  for  networks  with  more 
than  a few  hundred  nodes,  we  recognized  that  present  procedures  become 
prohibitively  expensive  because  of  the  large  amount  of  computer  time  and 
storage  needed  to  perform  the  optimization  step.  Design  procedures, 

* l 

based  also  on  a hierarchical  clustering  of  the  network  nodes,  have  been 
proposed  [FRAN  73],  [GERL  73B]  [C0V1  74]  to  substantially  reduce  the 

design  cost.  Again  consider  a set  of  nodes,  initially  not  connected  and 
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assume  that  we  wish  to  cluster  the  nodes  into  groups  which  themselves 
will  be  clustered  into  higher  level  groups,  and  so  on  up  to  a specified 
number  of  hierarchical  levels.  Once  the  hierarchical  classes  are  de- 
fined, then  the  previously  developed  network  design  techniques  for 
moderate  sized  networks  may  be  used  to  design  each  cluster  level  separ- 
ately. 

Several  questions  arise  as  to  the  optimal  clustering  structure, 
the  decomposition  of  the  global  performance  variables  and  requirements 
which  then  lead  to  a set  of  smaller  design  problems.  Frank  and  others 
[FRAN  72],  [FRAN  73]  showed  from  a feasibility  study  of  a 1000  node 
network  that  indeed,  hierarchical  structures  are  desirable  for  the  design 
of  large  networks.  They  also  posed  the  same  questions  concerning  the 


clustering  structure,  but  failed  to  answer  them  for  the  general  case  of 
an  arbitrary  number  of  hierarchical  levels.  Such  questions  will  be 
addressed  in  this  dissertation. 

1.3  Summary  of  Results 

The  above  considerations  demonstrate  the  need  for  some  new 
design  and  operational  (routing)  procedures  for  the  large  network  case. 
They  also  suggest  tha:  methods  based  on  some  sort  of  decomposition 
scheme  appear  to  be  desirable  for  both  the  routing  and  the  design  of 
large  networks. 

The  goal  of  this  dissertation  is  twofold.  First,  we  develop 
analytic  models  with  which  we  can  predict  and  optimize  the  performance 
of  a hierarchical  routing  in  large  distributed  networks.  Second,  we 
define  and  optimally  specify  the  deconposition  step  in  a hierarchical 
design  of  large  networks. 

Chapter  2 is  primarily  concerned  with  the  ii.  production  of  the 
hierarchical  routing  schemes  and  their  underlying  hierarchical  clustering 
structure  as  solutions  to  the  reduction  of  the  routing  information  and 
its  associated  overhead.  An  optimal  clustering  structure  is  found  to 
minimize  the  length  of  the  routing  table  and  consequently  it  results  in 
a minimum  cost  routing  scheme.  Enormous  gains  can  be  achieved  whereby 
the  table  length  may  be  reduced  from  N (N  ■ number  of  nodes)  to  e In  N. 
However,  a shortcoming  of  the  reduction  in  routing  information  is  the 
increase  in  the  path  length  of  a message  in  the  network. 

In  Qiapter  3.  we  examine  the  effect  of  hierarchical  routing  on 
network  path  length.  Bounds  are  derived  to  evaluate  the  maximum  increase 
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in  path  length  for  a given  table  reduction.  Hie  bounds  demonstrate  a 
major  result;  in  the  limit  of  a very  large  network,  enormous  table  re- 
duction may  be  achieved  with  no  significant  increase  in  network  path 
length. 

The  reduction  in  table  length  means  that  more  channel  capacity 
and  storage  are  available  for  the  transmission  of  data  traffic  in  the 
network.  However,  this  gain  in  communication  power  may  have  to  be 
partially  or  completely  paid  back  to  handle  the  excess  traffic  caused  by 
longer  network  paths. 

In  Chapter  4,  we  address  the  above  issue.  In  particular,  we 
are  interested  in  the  trading  relations  among  the  table  reduction,  the 
nodal  storage,  the  channel  capacity,  the  network  size,  the  throughput 
and  the  delay.  Several  queueing  models  are  developed  to  capture  and 
exhibit  the  interrelationships  among  these  variables.  The  models  demon- 
strate that  for  some  reasonable  cost  and  performance  constraints,  and 
for  a class  of  symmetrical  and  distributed  networks,  routing  procedures 
under  their  present  form  (non-hierarchical)  become  infeasible  for  a net- 
work size  beyond  some  "critical"  value;  on  the  other  hand,  hierarchical 
routing  with  an  optimally  selected  table  length,  preserves  a remarkably 
good  network  performance  for  a phenomenal  range  of  the  network  sizes. 

The  particular  numerical  examples  show  that  the  transition  point  where 
hierarchical  routing  becomes  certainly  better  than  a non-hierarchical 
one  occurs  for  a network  size  between  100  and  200  nodes. 

The  queueing  models  developed  in  this  chapter  represent,  in  a 
major  part,  an  extension  of  Kleinrock's  model  for  networks  [KLEI  64]. 
They  provide  us  with  some  new  results  concerning  the  effect  of  updates 


8 


and  storage  on  network  performance. 


In  Chapter  S we  address  the  following  two  issues:  (i)  the 

assignment  of  nodes  to  clusters,  clusters  to  superclusters,  etc.,  given 
an  arbitrary  network  and  clustering  structure;  (ii)  the  evaluation  of 
hierarchical  routing  as  applied  to  a more  general  network  environment 
such  as  the  ARPANET.  In  this  chapter  we  mainly  lay  the  framework  and 
introduce  a methodology  in  defining  nearness  measures  between  pairs  of 
nodes.  A clustering  technique  (Complete  Linkage  Method)  has  been  modi- 
fied in  order  to  fit  the  constraints  of  our  specific  network  environment. 
The  nearness  measures  and  the  clustering  technique  are  utilized  to 
achieve  the  clustering  of  the  June  1973  ARPANET. 

Furthermore,  the  simulation  of  a 64-node  torus  net  confirms  our 
theoretic  results  and  shows  that  even  for  such  a moderate  sized  network, 
an  appropriately  selected  hierarchical  routing  begins  to  exhibit  improve- 
ments in  network  performance  as  compared  to  a non-hierarchical  scheme. 

In  Chapter  6 we  address  some  issues  related  to  the  hierarchical 
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design  of  large  networks.  The  emphasis  is  on  the  determination  of  a 
clustering  structure  to  be  used  in  the  design  phase  and  which  minimizes 
the  computational  cost  of  the  design.  Such  a cost  is  assumed  to  grow 
exponentially  with  the  number  of  nodes  in  the  subnet  to  be  designed.  We 
present  optimum  results  both  for  the  number  of  clusters,  superclusters, 
etc. , and  the  number  of  hierarchical  levels  when  the  same  design  strategy 
(technology)  is  considered  at  all  levels.  Optimal  clustering  structures 
are  also  determined  when  different  design  strategies  are  considered, 
provided  that  the  number  of  levels  is  given.  An  expression  of  the 
average  delay  of  a message  in  such  a hierarchical  network  is  also 
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provided  in  tens  of  the  average  delays  in  the  layer  subnets  composing 
the  network.  This  decomposition  should  consequently  lead  to  the  design 
of  smaller  subnetworks  for  which  we  can  utilize  present  design  strategies. 

In  Chapter  7 we  give  some  concluding  remarks  and  suggest  topics 
for  further  research. 

Some  intermediary,  but  general,  results  were  required  which 
proved  to  be  quite  useful  in  the  analytic  performance  evaluation  of 
hierarchical  routing  in  large  networks  (Chaper  4) . These  results  form 
the  object  of  Appendixes  A and  B. 

In  Appendix  A we  derive  a closed  form  expression  for  the  average 
shortest  path  length  in  grid- type  networks.  Also  the  distribution  and 
the  corresponding  z-transform  of  path  lengths  in  a torus  network  are 
defined  and  determined. 

/ 

In  Appendix  B we  consider  various  schemes  for  sharing  a pool  of 
buffers  among  a set  of  communication  channels  in  a single  network  node 
environment  [KAMO  76] . Several  sharing  schemes  are  examined  and  the 
results  of  the  analysis  are  presented  and  displayed  in  a fashion  which 
permits  one  to  establish  the  tradeoffs  among  blocking  probability, 
utilization,  throughput  and  delay.  The  study  shows  that  no  one  scheme 
is  always  optimal,  and  that  the  selection  of  a particular  scheme  should 
fit  the  particular  application  considered.  In  general,  we  find  under 
normal  operating  conditions  that  sharing,  with  some  restrictions  on  the 
contention  of  space,  is  certainly  desirable,  especially  when  little 
storage  is  available. 

In  Appendix  C we  present  some  proofs  and  complementary  results 
for  Chapter  6. 
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Even  though  this  research  addressed  some  design  issues  which 
arise  in  large  computer  communication  networks,  the  methodology  devel- 
oped may  be  applicable  to  more  general  and  large  systems.  In  particular 
we  mention  a possible  application  to  packet-radio  networks,  where  clus- 
tering of  terminals  in  groups  may  alleviate  some  of  the  difficulties 
encountered  in  performing  the  routing  function. 

In  summary,  we  list  below  the  major  contributions  of  this 

research. 

(1)  We  characterize  and  evaluate  the  performance  of  hierar- 
chical routing  for  large  packet  switched  computer  networks;  models  are 
developed  to  determine  clustering  structures  which  lead  to  a minimal 
routing  cost  (storage,  capacity)  and  their  effect  on  the  network  path 
length.  More  importantly,  we  can  determine  the  effect  of  table  reduc- 
tion in  terms  of  network  throughput  and  delay.  As  a result,  we  are  able 
to  demonstrate  that  hierarchical  routing  is  remarkably  efficient  (in 
fact,  necessary)  for  large  distributed  networks. 

(2)  A general  methodology  is  specified  for  the  design  of 
large  hierarchical  networks.  A model  is  developed  to  determine  a parti- 
tioning structure  so  as  to  minimize  the  computational  cost  involved  in 
the  design  of  such  networks. 

(3)  Models  are  developed  to  study  the  effect  of  storage  in  a 
single  node  and  in  a network  environment.  Several  storage  sharing 
schemes  are  specified  and  analyzed  in  order  to  make  best  use  of  such  a 
resource  (storage) . 


I 


j 
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CHAPTER  2 

HIERARCHICAL  ROUTING  PROCEDURES 
FOR  LARGE  COMPUTER  NETWORKS 


The  most  commonly  used  distributed  adaptive  routing  policies 
base  their  decisions  on  routing  information  (delay,  hop  numbers) 
stored  in  tables  at  each  node  of  the  network.  These  tables  identify 
the  output  line  to  select  for  each  destination  in  the  network.  They 
are  also  regularly  updated  using  routing  information  collected  inter- 
nally (at  each  node)  and/or  provided  from  neighboring  nodes. 

For  large  computer  networks,  on  the  order  of  Many  thousands  of 
nodes,  the  routing  information  nay  become  excessively  costly  in  terms 
of  nodal  storage  required  to  keep  the  tables,  CPU  time  needed  to  per- 
form the  updates,  and  channel  utilization  incurred  by  the  exchange  of 
routing  information.  These  considerations  make  current  adaptive 
routing  policies  unattractive  for  large  networks. 

In  this  chapter,  first,  we  will  review  the  important  aspects  of 
adaptive  routing  procedures  for  computer  networks,  then  we  will  discuss 
general  problems  associated  with  the  i^iact  of  the  network's  size  on 
those  procedures.  Next,  routing  schemes,  based  on  hierarchical  parti- 
tioning  of  the  network,  are  presented  as  a solution  for  reducing  the 
amount  of  routing  information  and  the  associated  overhead.  The  focus 
of  this  chapter  is  to  determine  the  clustering  (partitioning)  structure 
which  results  in  a minimum  amount  of  routing  information  and,  conse- 
quently, in  a minimum  cost  routing  scheme.  Unfortunately,  the  gains 
obtained  from  such  schemes  are  acco^>anied  with  an  increase  in  the  path 
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length  of  a aessage  in  the  network  which  results  in  a degradation  of 
performance.  This  issue  will  be  the  object  of  the  next  chapter. 

2. 1 I ^ortant  Aspects  of  Adaptive  Routing  Schemes 

2.1.1  The  Routing  Problem  in  Networks 

In  a packet  switching  network,  messages  are  partitioned  into 
smaller  segments  called  packets  which  then  are  transmitted  through 
the  network  using  the  store-and-forward  technique.  That  is,  a packet 
traveling  from  source  S to  destination  D is  received  and  "stored"  in 
queue  at  any  intermediate  node  K,  while  awaiting  transmission,  and  is 
then  sent  "forward"  to  node  P,  the  next  node  on  the  route  from  S to  D, 
when  channel  (K,P)  permits  [KLEI  64],  [FULT  72],  [KAHN  72],  [MCQU  74] 
[KLEI  76]. 

The  selection  of  the  next  node,  P,  is  made  by  a well-defined 
decision  rule  referred  to  as  the  routing  policy.  Routing  policies  may 
be  divided  into  two  main  classes:  deterministic  and  adaptive  [GERL  73C], 

[FULT  72].  A deterministic  policy  is  time  invariant.  It  may  be 
selected  to  provide  the  optimal  routing  for  a network  in  steady  state, 
which  has  a given  traffic  requirement  governed  by  a certain  probabil- 
istic distribution  (see  Qiapter  6 for  more  details) , and  under  the 
assumptions  that  no  failures  can  occur.  The  optimal  policy  may  be 
determined  analytically  [FRAT  73]  and,  therefore,  it  is  attractive  to 
use  it  in  the  design  phase  [GERL  73A] . An  adaptive  policy  is  time 
varying  and  bases  its  decision  on  some  measure  of  the  observed  traffic. 

It  can  adjust  to  load  fluctuations  and  node  or  branch  failures.  For 
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such  a reason  it  is  more  attractive  for  real  network  operation  than 
a deterministic  policy.  However,  the  performance  evaluation  of  an 
adaptive  policy  requires  time-consuming  simulations,  which  makes  it 
inadequate  for  use  in  the  network  design  phase. 

2.1.2  Adaptive  Routing  Policies 

The  major  goal  of  an  adaptive  routing  procedure  is  to  sense 
changes  in  the  traffic  distribution  and  network  status  and  then  to 
route  messages  such  that  the  congested  areas  of  the  network  are  avoided. 
It  is  very  important  for  those  procedures  to  adapt  to  line  and  node 
failures  in  order  to  maintain  a good  grade  of  service  for  the  network. 
Such  policies  base  their  decisions  on  the  measured  values,  at  given 

times,  of  a set  of  tine  varying  variables  (maber  of  messages  enqueued, 

•• 

number  of  hops,  etc.)  which  describe  the  salient  features  of  the  state 
of  the  network  (traffic,  topology,  etc.).  Such  information  is  referred 
to  as  routing  information.  A central  node  could  provide  the  routing 
information  (centralized  control)  and  distribute  it  to  all  the  nodes 
in  the  network,  or  the  nodes  could  collaborate  in  computing  the 
routing  information  directly  (distributed  control)  [KLEI  64],  [FULT  72], 
[KAHN  72]. 

Specifically,  routing  information  is  stored  in  tables  at  each 
node  and  is  used  to  identify  the  output  line  for  each  destination. 

More  detailed  classifications  of  the  routing  policies  can  be  found  in 
[FULT  72],  [GERL  73C]  and  [MGQU  74].  In  this  study,  we  limit  our 
considerations  to  the  distributed  routing  policies  which  base  their 
decisions  on  routing  information  contained  in  routing  tables 
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individually  maintained  at  each  node.  The  tables  are  updated  peri- 
odically or  asynchronously  or  a combination  of  both  [FULT  72]  using 
routing  information  collected  internally  and  provided  from  neighboring 
nodes.  Such  a scheme  is  used  to  operate  the  ARPANET  [MCQU  74]. 

Typically,  in  a network  with  N nodes,  each  node  ("IMP"  in  the 
ARPANET  terminology)  i,  (i  * 1,  2,  . . . , N) , has  a routing  table  (to  be 
denoted  by  RT)  which  is  composed  of  N entries.  Each  entry,  say  k,  is 
subdivided  into  three  (or  more)  fields.  The  "delay"  field  indicates 
the  estimated  minimal  delay  from  node  i to  node  k.  The  "next -node" 
field  indicates  the  next  node  a message  must  be  forwarded  to  on  its 
way  to  node  k,  using  the  "minimal"  delay  path.  The  "hop"  field  rep- 
resents the  minimum  number  of  line  hops  to  node  k.  The  purpose  of  the 
hop-field  is  to  allow  the  detection  of  line  or  node  failures  in  the 
network. 

Each  node  periodically  (i.e.,  every  .64  sec  in  the  ARPANET, 
for  a heavily  loaded  50  kilobit  line)  sends  to  and  receives  update 
messages  from  neighboring  nodes;  these  updates  are  not  synchronized 
among  nodes.  Upon  reception  of  an  update,  a node  updates  its  own 
routing  table,  using  the  delays  measured  on  its  output  lines  and  the 
delay  information  found  in  the  update  message.  An  exasple  of  an  up- 
dating rule  is  provided  in  Chapter  5 and  is  used  in  a program  developed 
to  simulate  computer  networks. 

To  summarize,  let  us  say  that,  fundamental  to  the  operation  of 
the  distributed  adaptive  routing  schemes  is  the  storage,  the  propa- 
gation and  the  updating  of  routing  tables.  Also,  it  is  important  to 
note  that  in  such  schemes,  the  routing  tables  must  contain  a number  of 
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entries  equal  to  the  number  of  nodes  in  the  network. 

For  large  distributed  networks,  on  the  order  of  hundreds  or 
thousands  of  nodes,  the  adaptive  routing  policies  in  their  present  fora 
become  unsuitable.  This  comes  about  for  the  following  reasons: 

i.  The  excessive  amount  of  storage  required  at  each  node 

to  store  the  routing  tables  will  substantially  reduce  the  storage  avail- 
able for  other  functions  such  as  store- and- forward,  flow  control  and 
reassembly  functions.  Either  that  or  extra  nodal  cost  is  incurred  if 
more  storage  must  be  provided. 

ii.  The  large  CPU  overhead  required  for  the  maintenance  of 
the  routing  tables  (frequent  computations  of  the  best  routes)  will 
induce  either  large  delays  in  the  forwarding  of  messages  or  higher 
nodal  cost  for  faster  CPU's. 

iii.  If  no  added  capacity  is  provided,  the  channel  capacity 
required  for  the  transmission  of  the  routing  messages  could  become  so 
demanding  as  to  significantly  reduce  the  throughput  (load  carried)  of 
the  network. 

iv.  The  large  delays  incurred  in  the  propagation  of  the 
routing  information  throughout  the  network  will  reduce  the  degree  of 
adaptability  of  the  routing  scheme  to  changes  in  the  network.  In  order 
to  maintain  a good  response  time  in  avoiding  congested  areas  it  is 
necessary  to  increase  the  rate  of  updates  which,  consequently,  will 
require  more  CPU  and  channel  capaci ty. 

In  view  of  the  above  considerations,  routing  schemes  based  on 
a hierarchical  clustering  of  the  network  are  proposed  as  solutions  to 
keep  the  amount  of  routing  information,  hence  the  storage,  the  CPU  and 
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the  line  overhead  associated  with  it,  within  reasonable  limits  without 
deteriorating  the  performance  of  the  network.  With  respect  to  (iv)  , 
because  of  the  smaller  routing  tables,  it  is  possible  to  increase  the 
update  rate  without  much  extra  cost.  Fultz  [FULT  72]  and  McQuillan 
[MGQU  74]  proposed  similar  schemes  but  did  not  provide  any  quantitative 
analysis. 


2.2  m- level  Hierarchical  Routing  (HIR)  Schemes 

Because  of  the  underlying  hierarchical  partitioning  of  the  net- 
work, these  schemes  are  referred  to  as  the  MR  schemes. 

The  main  idea  in  reducing  the  table  length  (routing  information) 
is  to  keep  at  any  node,  say  i,  complete  routing  information  about  nodes 
which  are  close  to  i (in  terms  of  a hop  distance  or  some  nearness 
measure) , and  lesser  information  about  nodes  located  further  away  from 
i.  This  can  be  realized  by  providing  one  entry  per  destination  for 
the  closer  nodes,  and  one  entry  per  set  of  destinations  for  the  remote 
nodes.  The  size  of  this  set  may  increase  with  the  (average)  distance 
from  i to  the  nodes  in  the  set.  A similar  concept  underlies  the 
mechanisms  of  large  information  systems  with  pyramidal  structures, 
in  which  information  is  more  and  more  aggregated  as  we  move  up  to  the 
higher  levels  in  the  hierarchical  organization.  Aggregation  of  infor- 
mation or  variables  is  commonly  introduced  when  dealing  with  large 
systems  [MESA  70],  [CHUR  68],  [SCHO  71]. 

For  the  routing  in  large  nets,  the  aggregation  of  the  routing 
information  is  achieved  through  a hierarchical  partitioning  of  the 
network.  Basically,  an  m-level  hierarchical  clustering  (MIC)  of  a set 
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of  nodes  consists  of  grouping  the  nodes  (which  we  shall  define  as  0 
level  clusters)  into  1st  level  clusters,  which  in  turn  are  grouped  into 
2nd  level  clusters,  etc.  This  operation  continues  in  a bottom  up 
fashion  until  the  growing  of  the  m-2nd  level  clusters  into  m-lst  level 
clusters  whose  mion  constitute  the  mth  level  cluster.  The  mth  level 
cluster  is  the  highest  level  cluster  and  as  such  it  includes  all  the 
nodes  of  the  network.  The  >tiC  will  be  described  more  formally  in 
Section  2.3.1. 

Since  ItlR  schemes  are  based  on  an  m-level  hierarchical  cluster- 
ing of  the  nodes  of  the  network,  only  one  entry  in  the  routing  table, 
at  any  node,  say  i,  is  provided  for  each  node  in  the  same  1st  level 
clusters  as  i,  and  for  each  1st  level  cluster  (set  of  nodes)  in  the 
same  2nd  level  cluster  as  i,  and  in  general  for  each  k-lst  level 
cluster  in  the  same  k^  level  cluster  as  i (k  = 1,  2.,  ....  m) . The 
structure  of  this  scheme  can  be  best  understood  by  an  example.  Fig.  2.1 
shows  a 3-level  hierarchical  clustering  imposed  on  a 24  node  network. 

The  clustering  leads  to  the  tree  representation  shown  in  Fig.  2.2,  where 
nodes  are  identified  using  the  Dewey  notation  [KNUT  69  J . To  each  node 
is  now  associated  a reduced  routing  table.  Fig.  2.3  shows  the  entries 
of  node  1.1.1's  routing  table;  the  number  of  entries  is  now  10  instead 
of  24  without  clustering.  As  an  example,  the  routing  of  a packet  from 
node  1.1.1  to  node  3.2.2  may  proceed  as  follows:  Node  1.1.1  recognizes 

from  the  address  of  the  destination  node,  3.2.2,  that  it  has  to  use  entry 
3,  of  the  2nd  level  cluster  entries,  to  decide  upon  the  next  node  to 
which  the  packet  must  be  forwarded.  When  the  packet  reaches  a node,  say 
3.1.1,  in  the  2nd  level  cluster  3,  then  that  node  will  in  turn  use  the 
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Figur*  2.1.  A 3-L#**»  du«t*r*d  24-Nod*  Mftwortu 


second  entry  among  the  1st  level  cluster  entries.  Finally,  when  the 
packet  enters  the  destination  cluster,  3.2,  the  routing  will  be  done 
using  the  0th  level  cluster  entry  number  2.  Notice  that  it  was  assumed 
that  the  KHC  results  in  connected  sibgraphs. 

From  the  above  considerations,  the  following  problems  must  be 
resolved  at  the  outset: 

i.  The  determination  of  an  appropriate  clustering  structure, 
i.e.,  the  size  of  the  clusters  at  all  levels  and  the  number  of  levels 

ii.  The  definition  of  an  aggregate  routing  variable  for  the 
clusters  at  all  levels;  the  specification  of  new  updating  rules,  if 
necessary 

s t 

iii.  The  assignment  of  the  nodes  (0  level  clusters)  to  1 
level  clusters,  1st  level  clusters  to  2n<*  level  clusters,  etc. 

The  rest  of  this  chapter  will  focus  on  (i).  (ii)  and  (iii)  * 
are  respectively  treated  in  Chapters  3 and  5. 

Finally,  it  is  also  necessary  to  evaluate  the  performance  of 
the  MHR  schemes  as  compared  with  the  present  non-clustered  schemes. 

This  question  will  be  examined  in  various  ways  in  Chapters  3,  4,  and  5. 

2.3  Minimum  Routing  Information 

The  hierarchical  partitioning  of  the  network  has  as  an  objective 
the  reduction  of  the  size  of  the  routing  table.  It  is  then  iiqportant 
to  determine  the  specific  clustering  structure  (cluster  sizes,  number 
of  levels)  that  will  result  in  a minimum  table  length.  Hie  optimal 
sizes  found  will  serve  as  the  input  parameters  to  the  clustering  tech- 
niques (see  Chapter  5)  whose  function  is  to  assign  nodes  to  clusters. 
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clusters  to  superclusters,  etc.  In  what  follows,  we  will  introduce  the 
tree  representation  for  the  »«C  and  formally  pose  the  problem  of  find- 
ing an  optimal  clustering  structure.  We  will  then  proceed  with  the 
derivation  of  the  optimal  solution  and  the  study  of  its  characteristics 

2.3.1  Tree  Representation  for  the  WC  and  Problem  Statement 
Any  hierarchical  classification  scheme  lends  itself  to  a tree 

representation  [KNUT  69].  The  tree  structure  has  already  been  intro- 
duced in  Fig.  2.2,  to  represent  the  3-level  hierarchical  clustering  of 
the  24-node  network  in  Fig.  2.1.  The  representation  of  an  m-level 
hierarchical  clustering  is  shown  in  Fig.  2.4. 

2. 3. 1.1  Notation  and  Definitions 
Definitions : 

A level  cluster,  C^,  is  defined  recursively  as  a set  of 
s t 

k-1  level  clusters.  It  corresponds  to  a node  at  level  k,  in  the  tree 
representation  of  Fig.  2.4. 

A kth  level  cluster  is  identified,  similar  to  the  Dewey  notation 
[KNUT  69],  by  a vector  of  predecessors,  ik+J  = (im,  i^j,  ....  ik+J) 
which  can  subsequently  serve  as  an  address  of  C^.  The  index,  i^, 
indicates  the  m-lst  level  cluster,  say  CB  j(im),  to  which  belongs; 
i^j  indicates  the  m-2nd  level  cluster  in  (im!)  to  which  belongs; 
etc.  The  notation,  C,^,  ik+1)  or  (^(i^j),  will  be  used 

when  there  is  a need  to  identify  C^. 

Notice  that  a leaf  in  the  tree  representation  corresponds  to  a 
node  (0^  level  cluster)  in  the  network,  and  to  any  node  is  associated 


an  address  vector  ij  which  will  be  used  for  the  routing  of  messages. 

The  degree  of  a level  cluster,  C^,  is  defined  as  the  number 
of  k-lst  level  clusters  included  in  C^.  It  also  indicates  the  downward 
degree  of  the  corresponding  node  in  the  tree. 

Notation  and  further  definitions 


RT  = Routing  Table 

MIC  * m- level  Hierarchical  Clustering 
MIR  » m- level  Hierarchical  Routing 
S s Set  of  nodes  in  the  network 
N = size  of  the  above  set 
m = number  of  levels  in  the  hierarchy 

C^  = k**1  level  cluster,  represented  by  a node 

at  level  k in  the  tree  structure 

* 

Cq  * 0th  level  cluster,  network  node 
ik+i  * (iB»  •••»  ik+1)  - vector  of  predecessors 

Si^-k+l5  " S^m'  **•»  1k+l)  " co^leteiy  identified  k**1 
level  cluster 

nk^~k+P  * "k^m*  •**»  *k+P  * de8ree  tJie  ®bove  cluster 

n„  = degree  of  the  111th  level  cluster  - (root  of  tree) 
m 


|nk(ik+1)|  vector  of  degrees  of  all  kth 
~k+l  level  clusters 

(fij,  n2,  ...»  nj  degree  vector 


2. 3. 1.2  Expressions  for  the  length  of  the  Routing  Table  and  the  Site 
Constraint 

ct 

The  siumnation  of  the  degrees  of  all  the  1 level  clusters 
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gives  the  total  number  of  nodes  in  the  network  (i.e.,  the  total  nurter 
of  leaves  in  the  tree  structure.  Fig.  2.4).  Hence 


\(1.’ 


• ij^+j)  n2(*a»  •••»  £j) 


n jfijn*  • • • * ^2^ 


(2.1) 


Eq.  (2.1)  will  generally  serve  as  a constraint  over  the  choice  of  the 
optimal  degree  vector  and  it  will  be  referred  to  as  the  size 


constraint. 


Let  MCqC^)]  be  the  length  of  the  RT  at  node  C0(i^);  it  is 


defined  as  the  nuaber  of  entries  in  that  table.  Then 


t[C0(il)J  " "k^a*  hs*!5 


(2.2) 


The  assuaption  is:  each  node  of  the  network,  (^(ij),  contains  an  RT 

with  an  entry  for  each  k-lst  level  cluster  in  the  saae  kth  level  cluster 
as  C0(ii)  (there  are  n^Ci^,  •••»  ik+1)  such  entries),  and  this  for 
k ■ 1,  2,  . . .,  a. 

In  order  to  simplify  the  manipulation  of  the  RT’s,  we  will 
also  assume  that  equal  length  tables  are  provided  at  all  nodes.  Con- 
sequently, if  A is  that  length,  it  must  accomodate  the  number  of  entries 
in  the  RT  of  any  node. 


Hence 


i(m,  n)  « max  { £ n.  (i  , i , ....  i.  .)>  (2.3) 

(over  all  l(k>l  k ” a_1  k+1  ) 

\ nodes  / 
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2. 3.1. 3 Problem  Statement 


given 

minimize 


[see  Eq.  (2.3)] 


N 

A(m,  n) 
over  : m and  n , 

subject  to  : size  constraint  [see  Eq.  (2.1)] 

m a positive  integer  variable 

♦ 

n a vector  of  positive  integer  variables 


(2.4) 


2.3.2  Real- Valued  Solutiott  of  the  Optimization  Problem 

We  proceed  to  first  solve  Problem  2.4,  with  the  assumption  that 
U,  is  a real  valued  vector.  This  is  in  order  to  obtain  an  explicit 
analytical  expression  for  the  optiiiial  solution.  As  a consequence  of 


this  assuttptibn,  a stumnation  of  this  type  £ hj(i2)  becomes  meaningless 

i*»l 


if  n2  is  not  at  integer,  unless  all  the  h^ijj's  are  equal  (to  n1)  then 
the  summation  becomes,  h2n^.  In  fact,  the  solution  of  the  optimization 
problem  will  show  that  clusters  of  the  Same  level  must  be  of  equal 
degree,  hence,  all  the  Summations  in  Eq.  (2.1)  will  become  meaningful 
a posteriori. 


2. 3.2.1  Optimality  for  a fixed  a 
Proposition  2.1 

Given  m,  the  nuinber  of  levels  in  the  hierarchy,  and  assuming 
that  n is  a real  valued  vector,  the  solution  ofr  Problem  2.4  is  such 
that: 


t6 


. — 


The  foraulation  above,  is  equivalent  to 


i 


Let  n2  be  fixed.  Then,  summing  the  first  constraint  in  Problen  2.8, 
with  respect  to  n2,  we  find 


VI  1 T.  ni  ttj) 

V1 


(2.9) 


If  n is  feasible,  i.e.,  if  it  satisfies  Eq.  (2.1),  then  Eq.  (2.9) 
becones 


n > £-  + n,  ¥ n feasible. 
" **2  * 


The  above  equation  provides  a lower  bound  on  the  optimal 
fixed  n2<  Consequently,  if  a feasible  solution  achieves 
bound,  then  it  must  be  optimal.  Such  a solution  is 

n1^2^  * nj  i2  * 1*  2,  ...,  n2 

If  we  now  let  n2  be  a variable,  the  problen  reduces  to 

min  n » — ♦ V 
n2  2 

over  n2,  n2  ^ 0 
1/2 

whose  optimum  is  achieved  for  n2  « N ' which,  combined  with  Eq.  (2.10) 
proves  that  Proposition  2.1  is  true  for  m ■ 2. 

Assuming  that  Proposition  2.1  is  true  for  up  to  m - 1 levels, 
let  us  show  that  it  is  true  for  m levels.  The  size  constraint, 

Eq.  (2.1),  is  equivalent  to  the  following  set  of  constraints: 


solution  for  a 
that  lower 


(2.10) 
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Ut  “S  fix  the  variables  and  pdj,  - 1,  ...»  n^,  such  that 
Eq.  (2.12)  is  satisfied.  Problem  2.4  be cones  decomposable  into  n 
subproblems,  corresponding  eadi  to  a given  value  of  the  index  i^.  Such 
a subpvoblem,  for  a given  i » is 


max 

over  all  nodes 
in  same  m-lst 
level  cluster  C 


Eq.  (2.11)  must  be  satisfied 


From  the  induction  hypothesis,  the  solution  of  the  above  problem  is 
such  that 


objective  is  equal  to 
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If  we  now  let  the  pCi^'s  be  variable  but  we  keep  n^  fixed,  the  problea 


becomes 


■in:  max 

over:  pfi^) 


{",♦  (■  - UlpdJ]"'1} 


s.t.:  Eq.  (2.12)  Bust  be  satisfied  . 


The  above  problea  is  similar  to  Problea  2.7;  hence,  for  a fixed  n^, 
the  solution 


pCi,)  * — i,  * 1.  •••»  *», 


is  optiaal.  Consequently,  we  are  left  with 


■in  £ *»  (a  - 1)  — 
' J n 


over  n^,  n^  > 0 


Differentiating  £ with  respect  to  n , we  find 


(2.14) 


i - n-1  f„  r*/"'1 


Froa  the  above  equation,  we  determine  that  £ is  ainiaua  for 


n_  « N 


and  the  ainiaua  value  of  £ is 


(2.15) 


r-  ^ 


(2.16) 
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Substituting  Eq.  (2.15)  into  Eq.  (2.14),  then  Eq.  (2-14)  into  Eq.  (2.13) 
we  arrive  at  Eq.  (2.5).  Also,  Eq.  (2.16)  is  exactly  equal  to  Eq.  (2.6) 


2. 3.2.2  Global  Optimalit; 


So  far  we  have  solved  for  the  optimal  clustering  when  m,  the 
number  of  levels  is  fixed.  We  now  intend  to  let  m vary  and  conse- 


quently solve  for  the  global  optimum.  In  other  words,  we  intend  to 
solve  Problem  2.4  in  its  entirety  except  that  the  conqponents  of  the 
degree  vector  n and  the  number  of  levels  m,  are  assumed  to  be  real 


variables 


Proposition  2.2 


Under  the  conditions  of  Proposition  2.1  and  m being  a real 


variable,  the  global  optimum  clustering  is  achieved  for  a number  of 


levels 


The  corresponding  minimum  table  length  is 


Proposition  2.1  gave  us  the  optimal  clustering,  for  a fixed  m. 
Consequently  we  are  left  with  the  minimization  of  T(m),  Eq.  (2.6),  with 
respect  to  m.  Differentiating  Eq.  (2.6)  with  respect  to  m,  we  find 


d&Qn)  B /■  - In  N \ N 1/m 
dm  V m / 


The  above  equation  shows  that  J(m)  is  minimum  for  m as  given  in  Eq. 
(2.17).  Substituting  Eq.  (2.17)  into  Eqs.  (2.5)  and  (2.6),  we  arrive 
at  Eqs.  (2.18)  and  (2.19). 

'* 

2. 3.2. 3 Duality 

A simpler  formulation  of  problem  2.6  may  be  obtained  by  directly 
imposing  that  all  k**1  level  clusters  are  of  equal  degree,  n^ 

(k  = 1,  ...,  n).  Hence  Problem  2.6  becomes 


given: 

N 

min: 

m 

i ■ r nk 

over: 

k*l  k 

n = (n^,  ...,  n^) , m 

(2.20) 

s.t. : 

n \ - N 

k=l  K 

n,  m positive  integer  valued. 

It  can  be  (directly)  shown  by  induction,  that  the  real  valued  solution 
of  the  above  problem  verifies  Propositions  2.1  and  2.2. 

The  dual  formulation  of  Problem  2.20  is 


given: 

l 

max: 

N = 

over: 

n,  i 

■ 

~m 

. s.t. : 

E 

k=l 

a. 

i»l 


(2.21) 


n,  m positive  interger  valued. 
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The  objective,  in  the  formulation  above,  is  to  find  the  maxima  number 
of  nodes,  N , such  that  there  exists  an  MIC  whose  application  results 
in  a routing  table  of  length  The  dual  propositions  to  2.1  and  2.2 
are  respectively. 

Proposition  2.3 

For  a fixed  m and  l,  the  real  valued  solution  of  Problem  2.21 
is  such  that 

l 

n^  = ~ k * 1 , 2,  ...,m 

with  this  assignment 


Proposition  2.4 

The  real  valued  global  optimum  of  Problem  2.21  is  such  that 


Proposition  2.3  can  be  directly  proven  by  induction  on  the 
number  of  levels  in  the  hierarchy.  Proposition  2.4  is  a direct  con- 
sequence of  Proposition  2.3. 

2. 3. 2.6  Numerical  examples 

Figs.  2.5  and  2.6,  respectively,  illustrate  the  behavior  of 
T/N  and  (see  Eqs.  (2.6)  and  (2.19)  with  respect  to  m and  for 

several  values  of  N.  The  plots  exhibit  a flat  area  around  the  minimum. 
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They  also  show  an  initial  fast  decrease  of  T toward  a value  close  to 
the  minimum.  This  last  property. is  better  illustrated  in  Fig.  2.7 
where  A/N  is  plotted  with  respect  to  N for  m e {l,  2,  ...,  In  N}  ; 
this  indicates  that  most  of  the  table  reduction  can  be  obtained  with 
hierarchical  clustering  whose  nuiriber  of  levels  is  quite  a bit  smaller 
than  m*  (Eq.  (2.17)). 

■> 

2.3.3  Integer  Solution 

A 

In  this  paragraph  we  intend  to  solve  the  integer  optimization 
problem  as  formulated  in  2.20,  except  for  the  size  constraint  which  is 
changed  to  an  inequality,  the  problem  becomes 

m 

!min:  A = £ n. 

k*l  K 

over:  n,  m integer  valued  (2.22) 

m 

s.t. : II  n.  > N 

k=l  K 

The  above  modification  is  introduced  to  avoid  dealing  with  enpty 
feasible  sets  of  vectors  n,  for  some  values  of  m and  N.  A solution  n, 

m th 

such  that  n n.  > N,  only  implies  that  the  degree  of  any  k level 

k=l  K 

cluster,  k = 1,  . ..,  m,  must  be  less  than  or  equal  to  n^.  Practically 
this  means  that  there  will  be  vmused  entries  in  some  of  the  routing 
tables. 

Recall  that  the  global  optimum  real -valued  solution  is  such 
that  all  the  components  n^'s  are  equal  to  e,  therefore  in  the  integer 
case  we  are  not  surprised  that  the  following  proposition  holds  true. 
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Proposition  2.5 


There  exists  a global  optiaua  vector  n*  which  is  co^osed  of 
zero,  one  or  two  conponents  equal  to  2,  with  all  the  others  equal  to 


The  idea  is  that  any  number  (component  of  n)  or  set  of  numbers 


can  be  replaced  by  a set  of  2's  or  3's  which  results  in  the  same  sum 
but  a higher  product.  Hence  the  new  set  is  at  least  as  good  as  the 
original.  As  an  exaiqile,  we  list  typical  transformations. 


Original 

Transformation  Sum  Product 


New 

Product 


Original 

Numbers 


Let  us  now  proceed  with  the  proof  by  showing  that  we  can  always 
derive  from  an  optimal  solution  which  does  not  satisfy  Proposition  2.S, 
one  which  does.  Let  n*,  m*  be  a solution  which  does  not  satisfy  Prop- 
osition 2.5;  then,  at  least  one  of  the  following  conditions  is  true: 

(1)  at  least  one  component  is  equal  to  1,  (2)  at  least  one  component 
is  greater  than  3,  finally  (3)  at  least  three  coaponents  are  equal  to 


With  respect  to  (1)  let  us  prove  by  contradiction  that  this 


situation  cannot  occur.  Assume  that  n = 1,  then  consider  the 


This  idea  was  suggested  by  Dr.  D.  Cantor,  Mathematics  Department 
UCLA. 


vector  S’  composed  of  the  other  ■*  - 1 comments  of  i.e.. 


consequently 


«i  ■ \ k * 1.  ....  ■*  - 

vl  ■* 

n • n «v 

k-l  * k-1  11 


Hence  a'  is  a feasible  vector  which  results  in  a better  objective; 
that  is  a contradiction. 


Wjth  respect  to  (2),  assune  first  that  n _>  S,  then  consider 
the  vector  a'  composed  of  m,  ♦ 1 elenents  which  are  such  that 


("k  * "k  k-l,  i 

"*♦  " 3 

nvi " n«,  - 3 

With  the  above  transforation  the  objective  function  is  obviously  un- 
changed. The  product  is  such  that 


Wl  " + 1 + - 10 

hence  n 2 n'  n'  . > n +1 

■*  ■* 

■*+1  , /■*-!  \ ■* 

& ■y > a “k 

which  neans  that  a'  i*  *lso  feasible,  hence  optinal. 

* 4 then  *•  choose  n^  ■ n^+1  ■ 2 and  still  obtain 
s feasible  vector  a'.  The  above  operations  can  be  repeated  several 
tiMs  on  the  constructed  vectors  n'  in  order  to  transform  all  the 
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components  which  are  greater  than  3. 


With  respect  to  (3),  assume  that  n _ ■ n . ■ n ■ 2 ; 

----  ■***  •* 

then  consider  the  vector  nj  composed  of  m,  - 1 elements  which  axe  such 
that, 

^ * n^  k ■ 1,  • • • , m*  * 3 


n’  ~ * n'  , « 

n*-2  m*-l  * 


With  the  above  transformation  the  objective  fuction  is  unchanged.  The 
product  of  the  components  is  increased  by  1.  As  a result  is  also 
feasible,  hence  optimal. 

The  above  operation  can  be  repeated  several  times  on  the  con- 
structed vectors  n’,  until  we  reach  a solution  with  no  more  than  two 
2's. 

Finally,  we  can  see  that  the  repeated  application  of  the 
transformations  shown  for  (2)  and  (3)  will  eventually  generate  a vector 
n'  which  satisfies  Proposition  2.5. 

As  a consequence  of  Proposition  2.5  the  search  for  the  optimal 
number  of  levels  is  reduced  to  three  possibilities.  From  Problem  2.22, 
the  optimal  m,  must  be  such  that 


3®'X2X  > N 


where  x e {0,  1,  2} 

Hence,  the  three  possible  values  of  m are: 


1 


Finally  the  optical  m,  a*  , is  the  solution  of 


{■in  : l - 3n  - x 

over  (a, x)  e 0),  (Bj,  1),  (^,2)} 

Notice  that  the  optimal  pair  (m,x)  gives  the  conposition  of  the  optimal 
vector  n*. 

Proposition  2.6 

Given  a,  there  exists  an  optimal  vector  n which  is  such  that 
no  two  conponents  differ  by  more  than  1,  and  which  is  given  by, 


or  any  other  permutation  of  the  above  solution. 

Proof 

Let  u*  show  that,  given  any  two  numbers  which  differ  by  more 
than  1,  we  can  replace  them  by  exactly  two  numbers  which  do  not  differ 
by  more  than  1 and  which  result  in  the  same  siai  but  in  a better  product. 
TWo  cases  to  consider  depend  on  whether  the  difference  between  the  two 
numbers  is  odd  or  even: 

1/  (even),  pick  n and  n ♦ 2p. 

Their  sum  is  2n  ♦ 2p. 

2 

Their  product  is  n ♦ 2pn. 

Let  us  replace  then  by  n ♦ p and  n ♦ p;  then  the  sun  is  still  the 
same,  but  the  product  is 
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2 2 
n ♦ 2pn  ♦ p 

2 

which  represents  an  iaproveamt  of  p . A similar  proof  can  apply  for 
the  odd  case. 

Fro*  the  above  property,  we  conclude  that  any  k « 1,  ....  a, 

is  equal  to  a given  number,  either  a or  a ♦ 1.  If  we  let  x represent 
the  nuaber  of  components  equal  to  a ♦ 1,  then  the  problem  reduces  to 
/■in  l ■ (a  - x)a  ♦ x(a  ♦ 1)  = aa  x 
/ over  (a,  x) 

( s„t.  a*"x(a  ♦ l)x^  N 

a positive  integer;  x_<  a,  positive  integer. 

Let  us  show  that  there  exists  at  least  one  coaponent,  say  n^,  equal  to 
N1^*.  Froa  the  constraint  above,  the  optimal  a is  such  that 

(i)  x - 0 ->  a*  > N ->  a « Fn1/b1 

(ii)  x ^ 0 ->  (a  ♦ 1)“  > (a  ♦ 1)VX  > N ->  a ♦ 1 - Fn1/b1 

Knowing  that  n#  = Fn^b1,  Problem  2.22  can  be  reduced  to  a - 1 

variables,  with  N replaced  by  N/n^.  Froa  the  saae  considerations  as 
above,  we  know  that  the  optimal  vector  (nj,  n2,  . ..,  n^  ) for  the  new 
problea  has  at  least  one  conponent,  say  n^j,  equal  to 


Repeating  the  saae  operation,  we  arrive  at  Eq.  (2.23). 

Numerical  exaaples 

Similar  curves  as  in  Fig.  2.S  are  plotted  in  Fig.  2.8.  They 


42 


illustrate  the  behavior  of  A (Ad)  versus  m for  the  integer  case.  They 


exhibit  the  properties  described  in  Section  2. 3. 2. 4,  mainly  the  enormous 


table  reduction  for  small  values  of  a.  By  comparing  Fig.  2.5  with 


Fig.  2.8  we  notice  that  the  integer  valued  solution  is  extremely  close 


to  the  real-valued  solution.  Consequently,  we  will  limit  any  further 


considerations  to  the  siiqple  real-valued  solution. 


Fig.  2.9  illustrates  the  behavior  of  Nj/N,  where 


Nd " .n 


Notice  that  the  slack  in  the  size  constraint  is  relatively  small  for 


large  values  of  N. 


2-2 3-4  Optimality  with  No  "Self -Entries"  in  the  Routing  Table 


In  the  previous  model,  at  each  routing  table,  one  entry  (to  be 


called  self-entiy)  is  reserved  for  the  node  which  contains  that  table. 


and  one  for  each  of  the  kth  level  clusters  k = 1,  2,  ....  m - 1,  to 


which  that  node  belongs.  For  some  MHR  schemes  (e.g.,  those  defined 


in  Chapter  3)  and/or  with  some  extra  CPU  overiiead,  the  updating 


algorithm  can  operate  without  those  self-entries.  Consequently,  the 


new  length  A'  of  the  RT's  is. 


A - m 


(2.24) 


where  A is  given  by  Eq.  (2.3). 


The  optimal  clustering  structure  is  the  solution  of  Problem 


2.4  where  A is  replaced  by  A'. 


Real  Valued  Solution 


For  a fixed  m,  Eq.  (2.5)  still  holds  true.  Hence  the  mini 
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length  is 


V = irtT 


In  order  to  find  the  global  optimum,  let  us  differentiate  Z'  with 


respect  to  m. 


£■  (.  - - 1. 


Taking  the  2 derivative,  we  find 


d2!'  fin  N)2  „l/m  • 

*•>  “ 7 W 


For  m > 0,  — is  positive;  hence  is  a non-decreasing  function 
dm 

of  n.  Consequently  it  is  not  greater  than  its  value  at  m « •,  which 
is  equal  to  zero.  We  conclude  that  1'  is  a decreasing  function  of  m 
and  that  it  reaches  its  minimum  for  m equal  to  infinity. 


Z*  = limit  n (N  - 1)  = In  N 


In  summazy,  the  global  optimum  is  such  that 


m!  = + oo 
* 


z;  * In  N 


k > 1 . 


The  above  result  is  to  be  compared  with  Eq.  (2.19), 


which  indicates  that  theoretically,  an  improvement  of  a fraction 

e 

of  the  global  minimum  length  can  be  obtained.  Ihese  limiting  results 
are,  however,  meaningless  in  the  integer  case. 


Integer  Valued  Solution 

For  a fixed  m,  Proposition  2.6  still  holds  true.  As  for  the 
global  optimua  the  problem  to  solve  is. 


min:  T n.  - m 
k*l  K 

over:  n,  m integer  valued 


(2.25) 


s.t. : II  > N 

k*l  K ~ 

Recall  that  the  real  valued  solution  is  such  that 

* limit  * 1+ 

m -*■  <*> 

Therefore  we  are  not  surprised  that  the  following  proposition  holds 
true. 

Proposition  2.7 

There  exists  a non-degenerate  (i.e.,  no  one  conponent  is  equal 
to  one)  global  optimum  vector  n*  which  is  such  that 


k * 1,  2,  ...,  m* 


™ fin  Nl 

a | ITT  I 


(2.26) 


(2.27) 


Proof 

Let  us  first  show  that  if  ji*  is  an  optimal  vector  which  contains 

at  least  one  component  equal  to  1,  then  the  vector  obtained  after 

deletion  of  all  the  l’s  is  also  optimal. 

Assume  that  n ■ 1,  then  consider  n'  such  that 
■*  ~ 

k ■ 1,  ...,  m*  — 1 . 
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Notice  that  the  table  length  and  the  product  of  the  coaponents  are  un- 
changed. Thus  n*  is  also  feasible,  hence  it  is  optimal.  The  above 
operation  can  be  repeated  several  times  until  we  eliminate  all  the 
components  which  are  equal  to  one.  In  the  sequel  we  restrict  our  con- 
siderations to  non- degenerate  solutions  (i.e. , no  one  component  is  equal 
to  one) . 

Let  ust  first  prove  this  intermediary  result. 


Fact: 


If  n*  is  a non- degenerate  optimal  vector  then  it  must  be 
composed  of  zero,  one  or  two  components  equal  to  3 with  all  the  others 
equal  to  2. 

Proof 

The  proof  proceeds  by  contradiction.  Assume  that  n*  is  a 
non-degenerate  optimal  vector  which  does  not  satisfy  the  above  fact. 

It  must  be  such  that  at  least  one  of  the  following  conditions  is  true; 
(1)  at  least  one  component  of  n*  is  greater  than  or  equal  to  4;  (2)  at 
least  three  components  of  Q*  are  equal  to  3. 


With  respect  to  (1),  assume  that  n^  >_  4,  then  consider  the 


vector  fl' . 


("i  * \ 

(n-  . 2 


where 


"0 


is  such  that 


k * 1,  2,  ...»  m*  - 1 

i = m*,  m*  ♦ 1 ■*  ♦ «o 


*»<„  <2vl 


(2.28) 


(2.29) 


Thus 


B*+“o 

n 

k«l 


nk-  2 


®n+1  m*-1 


m* 


n ni>  n nk 

k=l  K k*l  K 
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~n  irnrurra-iTJWryriFM  i ■W»  Tlf <yf|pi'i  ;mh 


Consequently  a,'  is  feasible.  Also  the  objective  function  is 


m*+m0 


£ "k  - C".  * V ■ 3l  "k  * 2<»0  ♦ » - <">)  * *.)• 

i ' 

If  we  subtract  the  above  table  length  from  the  one  obtained  with  n*, 
then  the  difference  is 


4 “ V >o  - 2 


(2.  JO) 


If  A > 0,  then  we  reach  a contradiction.  Let  us  prove  that  in  •'fact 


A > 0. 


i.  n * 4 m_  * 1 — > A = 1 


ii.  n > 5 -<k  m.  > 2 . 

®*  “ 0 — 

It  can  easily  be  shown  by  induction  that 
®n 

2°>m0+l  Vm0  > 2 

From  Eqs.  (2.29)  and  (2.30), 

“o  mn 

A > 2 u • a - 2 > 2 u ■ » - 1 > 0 

o — o 

Thus  A is  always  greater  than  zero,  which  is  a contradiction. 
With  respect  to  (2).  assume  that 


V2  = nm.-l  = n.#  ' 3 

consider  the  vector  a'  composed  of  m*  + 2 elements, 

!nk  k » 1 , . . . , m*  - 3 

2 k » m#  - 2,  ...,  m*  + 2 

It  is  obvious  that  n'  is  feasible  and  that  it  reduces  the  table  length 
by  one,  which  contradicts  the  fact  that  n*  is  optimal.  Q.E.D. 


End  of  Proof  of  Proposition  2.7 

The  rest  of  the  proof  is  a consequence  of  the  above  fact.  Let 
n*  be  a non- degene rate  optimal  vector.  If  j\#  does  not  satisfy  Prop- 
osition 2.7,  then,  because  of  the  above  fact,  it  must  contain  one  or 
two  3's.  Any  3 can  be  replaced  by  two  2's,  without  changing  the 
objective  function  and  still  keeping  feasibility.  As  a result  we  arrive 
at  a vector  composed  only  of  2's  and  for  that  vector  to  be  optimal  the 
number  of  components  must  be  a solution  of 

/ min:  m 

/ over:  m integer 
( s.t. : 2*  > N 

The  solution  of  the  above  problem  is  given  by  Eq.  (2.27),  which  term- 
inates the  proof. 

2.4  Conclusions 

This  chapter  was  primarily  concerned  with  the  introduction  of 
the  MIR  schemes  and  their  tnderlying  hierarchical  clustering  structure 
as  solutions  to  the  reduction  of  the  routing  information  and  its 

■v  . 

associated  overhead.  We  found,  indeed,  that  enormous  gains  can  be 
obtained  whereby  the  length  of  the  routing  tables  may  be  reduced  from 
N entries  to  the  order  of  e-ln(N)  entries.  However,  a shortcoming  of 
these  gains  is  the  increase  in  the  path  length  of  a message  in  the  net- 
work. This  comes  about  from  the  fact  that  a given  node  must  send  all 
its  traffic  to  a given  cluster,  on  the  same  path  to  that  cluster.  This 
path  will,  in  general,  be  optimal  only  for  a subset  of  the  nodes  in  the 
destination  cluster.  Consequently,  some  messages  wijl  follow  longer 


I 

paths  than  they  should.  This  issue  will  be  addressed  in  the  next 
chapter. 

It  is  also  possible  that  less  routing  adaptability  could  result 
from  the  MIR  schemes  because  of  the  aggregation  of  the  routing  informa- 
tion. This  fact  may,  however,  be  beneficial  in  our  context  of  large 
networks  where  the  routing  policy  need  not  adjust  to  very  remote  and 

probably  short  lived  fluctuations. 

* 


CHAPTER  3 


STATIC  EVALUATION  OF  HIERARCHICAL  ROUTING: 
BOUNDS  ON  THE  INCREASE  IN  NETWORK  PATH  LENGTH 


Built  into  the  hierarchical  routing,  proposed  in  Chapter  2,  is 
the  reduction  of  routing  information.  As  a result  there  will  be,  in 
general,  an  increase  in  the  network  path  length.  The  magnitude  of  the 
increase  is  closely  related  to  the  length  of  the  routing  table  and  to 
the  choice  of  a specific  aggregate  routing  variable. 

In  this  chapter,  two  hierarchical  routing  schemes  and  a non- 
clustered (non-hierarchical)  scheme  are  presented,  and  their  routes  are 
characterized  under  some  "equilibrium"  conditions.  Then,  with  some 
assumptions  on  the  network  topology  and  on  the  partitioning  structure, 
bounds  on  the  increase  in  the  network  path  length  are  derived.  These 

bounds  are  expressed  and  studied  in  terms  of  the  relative  table  length 

* 

(t/N) . They  demonstrate  an  asymptotic  result  for  a class  of  large 
networks.  The  result  shows  that  when  the  number  of  nodes  grows  to 
infinity,  enormous  table  reduction  can  be  obtained  with  a relatively 
insignificant  increase  in  the  network  path  length. 

3. 1 Path  Characteristics  for  Hierarchical  and  Non-Hierarchical 
Adaptive  Routing  Policies 

The  purpose  of  this  section  is  to  characterize  the  actual  or 
virtual  routes  obtained  from  the  routing  tables  under  certain  equilib- 
rium conditions  as  defined  below.  The  routing  schemes  are  assumed  to 
belong  to  the  class  of  hierarchical  or  non-hierarchical  adaptive 


policies,  considered  in  Chapter  2.  Such  policies  basically  propagate 
routing  information  describing  the  length  of  the  paths  to  reach  any 
destination  node  or  a set  of  nodes.  The  path  length  is  defined  as  the 
sum  of  the  lengths  of  all  the  channels  which  constitute  that  path. 
Moreover,  the  length  of  a given  channel  is  usally  a random  variable 
which  may  reflect  the  utilization  and/or  the  excess  capacity  and/or  any 
other  information  which  partly  or  entirely  describes  the  stochastic 
state  of  that  channel.  The  transient  nature  of  adaptive  routing  renders 
the  analysis  of  the  above  problem  extremely  conplicated.  In  order  to 
make  any  progress  we  will  assume  that  all  channels  are  of  constant 
length.  This  is  a simplifying  assumption  which  will,  however,  allow  us 
to  capture  the  effect  of  clustering  on  the  network  path  length;  this  is 
the  main  objective  of  this  chapter.  Morever  the  above  assunption  is  an 
accurate  description  of  routing  policies  which  are  only  sensitive  to 
changes  in  the  network  topology,  and  of  more  general  policies  operating 
under  light  traffic  conditions  [KLEI  64].  Furthermore  if  all  the  chan- 
nels are  considered  to  be  of  equal  length  (say  1) , then  the  routing 
information  is  simply  what  we  defined  earlier  as  the  hop  distance. 

Such  routing  information  is  in  general  utilized  by  routing  policies  to, 
at  least,  detect  changes  in  the  network  topology. 

In  summary  we  will  restrict  our  considerations  to  hierarchical 
or  non-hierarchical  routing  schemes  (also  referred  to  as  clustered  and 
non-clustered  routing  schemes]  which  use  as  routing  information  the 
path  length  only.  Also  we  consider  that  all  channels  are  of  constant 
length.  In  what  follows  we  first  assume  that  all  channels  are  of  equal 


length  (one  hop)  and  then  we  generalize  to  arbitrary  (constant)  length 
channels. 

3.1.1  Non-Clustered  Routing  (NCR) 

Recall  that  the  NCR  schemes  operated  with  complete  routing 
information,  i.e.,  the  routing  tables  contain  one  entry  per  destination 
node  in  the  network.  Because  of  the  frequent  exchange  of  routing  in- 
formation between  neighboring  nodes,  the  routing  tables  will  in  general 
indicate  the  next  node  in  the  shortest  path  to  any  destination.  This 
is  due  to  the  fact  that  at  each  update,  a node  conpares  its  own  routes 
to  the  neighbor's  routes  and  updates  its  table  by  keeping  the  shorter 
routes.  The  above  approach  is  in  essence  a dynamic  search  for  the 
minimal  paths  in  the  network  [HU  69],  [FRAN  71].  As  a consequence, 
after  a certain  number  of  updates  (less  than  or  equal  to  the  diameter 
of  the  network)  if  no  changes  occur  in  the  network  topology,  all  the 
tables  will  indicate  exactly  the  shortest  paths.  If  a change  in  topol- 
ogy occurs,  then  some  of  the  paths  will  not  be  optimal  during  a 
transitory  period  (time  for  the  routing  information  to  percolate  to  the 
concerned  areas  in  the  network) . However  for  the  purpose  of  comparison 
with  the  hierarchical  routing,  we  will  assume  that  the  tables  always 
carry  the  shortest  path  information.  A more  formal  treatment  of  the 
above  is  presented  within  the  context  of  the  clustered  schemes  since, 
as  we  will  see  in  the  sequel,  a 1-level  hierarchical  routing  is  equiv- 
alent to  an  NCR  scheme. 
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3.1.2  Specification  of  the  WR  Schemes 


Built  into  the  MIR  schemes,  considered  in  Qiapter  2,  is  the 
reduction  of  the  routing  information  whereby  one  entry  in  a routing 
table  may  be  reserved  for  more  than  one  destination  node.  Routing 
information  is  aggregated  whenever  it  is  exchanged  between  special 
nodes  in  different  clusters  at  any  level.  Such  special  nodes  will  be 
referred  to  as  exchange  nodes.  IVro  MIR  schemes  will  be  presented  below. 
They  differ  only  in  the  definition  and  subsequently  the  computation  of 
the  aggregate  routing  information.  Ihe  two  schemes  will  be  referred  to 
as  the  Closest  Entry  Routing  (CER)  and  the  Overall  Best  Routing  (OBR) 
schemes.  In  order  to  proceed  with  their  description,  we  first  need  to 
specify  the  underlying  m-level  hierarchical  partitioning  of  the  set  of 
nodes  of  the  network. 

Assumption  3.1 

The  underlying  MIC  structure  of  the  set  of  network  nodes  is 
such  that  all  clusters  at  the  same  level  k,  are  of  equal  degree, 
n^,  k * 1,  ...,  m.  Also  the  subset  of  nodes  composing  a cluster  at 
any  level  and  their  incident  channels  constitute  a 1-connected  (at 
least  one  path  exists  between  any  pair  of  nodes)  cluster  subnetwork. 

The  former  property  of  the  above  assumption  partly  satisfies 
Proposition  2.1  which  defines  the  optimal  clustering  structure  that  we 
will  eventually  use.  The  latter  property  is  necessary,  since  the  traf- 
fic exchanged  between  nodes  in  the  same  cluster  must  follow  paths 
included  in  that  cluster's  subnet. 

Because  of  the  above  assumption  the  notation  of  Qiapter  2 


can  be  greatly  simplified.  Particularly,  the  degree  vector  is  reduced 
2 ■ (»|i  »2*  •••»  "J-  Moreover,  if  there  is  no  need  to  identify 
a cluster  with  its  entire  address  vector,  then  a shorter  notation, 
such  as  below,  can  be  used. 

Cfcfj)  ^ entry  in  k**1  level  cluster  entries 
°r  th 

Cfcfs)  A k level  cluster  containing  an  arbitrary  node  s. 


As  a consequence  of  Assumption  3.1,  the  routing  tables  at  any 
node  will  have  the  format  shown  in  Fig.  3.1,  and  will  contain 
t • nt  + n2  + ...  ♦ n^  entries.  Notice  that  self  entries  are  included 
in  the  routing  table  (RT) . The  self  entries  of  the  RT  at  an  exchange 
node  may  be  assigned  to  carry  the  aggregate  routing  information  from 
one  cluster  to  another.  The  content  of  the  self  entries  in  tables  at 

other  nodes  (non-exchange  nodes)  need  not  be  specified  in  this  study. 
TVo  aggregation  procedures,  each  pertaining  to  a particular  MHR  scheme 
(OBR  or  CER),  are  presented  below.  Also,  possible  ways  of  implementing 


those  procedures  are  suggested  with  the  assumption  that  neighboring 
nodes  exchange  their  RT's  as  routing  updates  (is  mentioned  in 
Chapter  2) . 


tion  of  Routing  Information 


i.  With  respect  to  the  CER  (Closest  Entry  Routing)  scheme, 
no  routing  information  describing  the  internal  behavior  of  a cluster 
is  propagated  outside  the  cluster.  With  this  rule,  a cluster  is  re- 
garded from  the  outside  as  a single  (super-)  n »de  whose  distance  to 
itself  is  equal  to  zero.  A way  of  implementing  this  scheme  is  to  assign 
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(HOP  FIELD) 


NEXT  PATH 
NODE  LENGTH 


C0(s) 

0th  LEVEL  CLUSTER 
SELF  ENTRY 


n,  0™  LEVEL  CLUSTER 
ENTRIES 


c0(i) 

jth  ENTRY  IN  0th 
LEVEL  ENTRIES 


Ck.1<*>  . 

k-18*  LEVEL  CLUSTER 
SELF  ENTRY 


nk  k-18*  LEVEL  CLUSTER 
ENTRIES 


Ck-ltt) 

Ith  ENTRY  IN  k-18* 
LEVEL  ENTRIES 


nk+.,  k1"  LEVEL  CLUSTER 
ENTRIES 


nm  m-18*  LEVEL  CLUSTER 
ENTRIES 


- 


I 

I 


zero  as  the  content  of  all  the  self  entries  at  any  RT  located  at  an 
exchange  node.  In  other  words,  the  distance  fro*  an  exchange  node  to 
the  clusters  at  all  levels,  to  which  it  belongs,  is  considered  to  be 
equal  to  zero. 

ii.  As  for  the  OBR  (Overall  Best  Routing)  scheme,  the  average 
estimated  distance  from  an  exchange  node  to  all  the  nodes  in  its  cluster 

(including  itself)  will  be  propagated  when  necessary  as  the  aggregate 

. 

routing  information  relative  to  that  cluster.  A possible  implemen- 
tation is,  at  any  exchange  node's  RT,  to  assign  to  the  hop  field  of  the 

k**1  level  cluster  self  entry  the  average  of  the  contents  of  the  hop 

st 

fields  of  all  the  k-1  level  cluster  entries  in  that  RT.  When  re- 
quired, the  confutation  of  the  averages  must  proceed  sequentially, 
starting  from  kcltok*m-l  (see  Fig.  3.1).  The  hop  field  of  the 
0th  level  cluster  self  entry  is  always  set  to  zero.  The  interpretation 
of  the  above  rule  is  that,  at  any  exchange  node,  say  e,  the  1 level 

cluster  self  entry  of  the  RT  indicates  the  estimated  average  distance 

s t 

from  node  e to  any  node  in  the  same  1 level  cluster  as  e,  including  e. 
In  general,  the  kth  level  cluster  self  entry  indicates  the  estimated 
average  distance  (in  hops)  from  node  e to  any  node  in  the  same  kth  level 
cluster  as  e,  including  e. 

Notice  that  the  content  of  the  self  entry  fields  can  be  well 
computed  by  the  exchange  node  receiving  the  routing  update  or  by  the 
one  generating  the  update.  This  fact  will  become  clear  in  the  example 
treated  right  after  the  specification  of  the  update  rule. 

Update  rule 

Let  s and  t be  two  neighbor  nodes  (i.e.,  they  are  connected 
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by  a channel  (s,  t))  which  belong  to  the  sane  level  cluster  and 
not  to  any  lower  level  cluster,  (k  = 1,  2,  ...»  m).  Let  1 (s)  and 

st 

Cjc_ j Ct)  respectively  denote  the  k-1  level  clusters  to  which  s and  t 
belong.  From  the  condition  above,  we  know  that 

Ck-l(s)  * Ck-l(t)  c S 311(1  Si-l(s)  n Sc-l^5  3 * 

As  a consequence,  and  due  to  the  RT  organization  as  specified  in  Chap- 
ter 2,  the  routing  tables  at  s and  t are  such  that  all  the  p-level 
cluster  entries  for  p = 0,  1,  ...»  k - 2 refer  to  different  cluster 
destinations,  whereas  all  the  other  entries  refer  to  the  same  cluster 
destinations. 

The  object  of  the  updating  procedure  is  to  compare  the  esti- 
mated lengths  of  the  paths  from  s or  t to  any  common  destination.  Then, 
the  routing  tables  are  updated  to  show  the  better  paths.  Moie  formally, 
let 

Cj (i)  i * 1,  2,  ...»  n^+1;  j * k - 1,  ...,  m - 1 

denote  a common  (to  s and  t)  jth  level  cluster. destination.  To  that 
cluster  is  associated  entry  i (at  both  tables)  among  the  jtl1  level  clus- 
ter entries;  that  entry  will  also  be  denoted  by  C^(i)  (F^g.  3,1).  Also 
let  HF(u,  C^(i))  represent  the  content  of  the  hop  field  of  entry  C^(i) 
at  node  u (u  » s or  t) . Finally,  whenever  node  t receives  an  update 
message  from  node  s,  then  for  each  common  destination  entry  (i)  the 
following  updating  algorithm  is  performed. 


IF  HF(t,  Cj(i))  > 1 ♦ HF(s,  Cj(i)) 

THEN  HF(t,  C.(i))  + 1*  HF(s,  C..(i)) 

NEXT  NODE  FIELD  OF  C. (i)  «-  s 


u: 
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Initially  all  the  entries  are  set  to  a large  value  (®) , except  for  the 
self  entries.  If  a CER  is  used  then  all  the  self  entries  are  set  to 
zero,  and  if  an  OBR  is  used  then  only  the  0tl*  level  cluster  self  entries 
are  set  to  zero,  e.g. , at  node  s 


HF(s , Cq(s))  A HF(s,  s)  = 0 


HF(s,  (ys)) 


0 CER 


» OBR 


k=  1,  ...,  m-  1 


all  other  entries  = ® 


Notice  that  in  the  algorithm  above,  it  is  assumed  that  all  the  routing 
information  contained  in  the  non-common  destination  entries  in  node  s 
routing  table  is  aggregated  as  specified  before,  to  represent 
HF(s,  C^ys)).  Moreover  the  content  of  the  common  self  entries  is  not 
relevant. 

The  fact  that  HF(s,  C^^Cs))  need  only  be  coagmted  for  updating 
purposes,  makes  it  clear  that  either  the  sending  (s)  or  the  receiving 
node  (t)  can  perform  the  aggregation  of  routing  information. 

A few  more  remarks  can  be  stated  about  the  above  updating  rule. 

st 

i.  If  s and  k belong  to  the  same  1 level  cluster,  then 
their  RT's  contain  only  common  destination  entries.  As  a result. 
Algorithm  3.1  will  be  performed  for  all  the  entries  in  the  table. 

ii.  A unique  "degenerate"  MIR  routing  scheme  corresponds  to 
either  the  OBR  or  the  CER  schemes  with  only  one  hierarchical  level,  and 
it  is  exactly  the  non-clustered  scheme  specified  above.  Moreover,  for 
such  a degenerate  case  all  the  network  nodes  belong  to  the  same  unique 

St 

1 level  cluster;  hence,  as  expected,  the  updating  algorithm  will  be 
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performed  for  all  the  entries  in  the  RT's. 

iii.  For  any  pair  of  nodes  s,  t,  the  common  region  in  the  rout- 
ing tables  can  be  determined  by  inspecting  the  address  vectors  of  s and 
t.  A similar  operation  is  required  by  the  routing  function  described 
later  in  this  section. 

We  are  now  ready  to  treat  an  exaople  which  will  expose  the 
mechanisms  of  updates  and  aggregation. 

Example  of  update 

Let  us  consider  the  14  node  network  shown  in  Fig.  3.2. 


If  no  clustering  is  considered  then  each  node  contains  a 14-entry  rout- 
ing table.  Numbers  from  1 to  14  are  assigned  to  the  nodes  to  serve  as 
addresses  (shown  on  the  right  of  each  node  in  Fig.  3.2).  If  a hierar- 
chical routing  is  used  then  the  underlying  clustering  structure  is 


I 


. .,.4  . 


composed  of  2 clusters  C^(l),  Cj(2)  containing  7 nodes  each,  i.e., 
nl  * 7*  n2  = 2*  * * 2 (see  3.2).  The  node  addresses  are  as  de- 


fined in  Section  2. 3. 1.1,  and  they  are  shown  on  the  left  of  the  nodes 


in  Fig.  3.2.  With  this  clustering  structure  the  length  of  an  RT  is 


reduced  to  9. 


In  what  follows  we  will  present  the  outcome  of  the  updates  and 


a few  key  steps.  The  initialisation  is  performed  as  mentioned  above. 


After  a certain  number  of  updates,  if  an  NCR  is  used,  all  the  RT's  will 


show  (in  the  hop  field)  the  shortest  distances  to  all  the  nodes  in  the 


network,  and  the  first  nodes  of  the  corresponding  shortest  paths  in  the 


"next -node"  field.  A typical  RT  is  shown  in  Table  3.1. 


Destination  j Next  Node  | Hop  Field 


Node  12,  NCR  Used 


Table  3.1  RT  at  Node  12 


B 


If  an  OBR  or  a CER  is  utilized,  then  updates  between  nodes  in 

the  same  cluster  are  performed  similar  to  the  non-clustered  case.  As  a 

consequence,  after  a certain  number  of  updates  the  RT's  will  show  the 

shortest  internal  (included  in  the  cluster)  paths  for  nodes  in  the  same 

cluster.  Tables  3. 2. a and  3.2.b  show  the  outcome  of  the  updates  at  the 

exchange  nodes  routing  tables.  For  the  sake  of  clarity,  the  contents 
St 

of  the  1 level  cluster  self  entries  are  not  shown,  and  also,  the 
other  cluster  entries  are  left  equal  to  the  initial  value  (<*>) . 
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1 


Destination 


(1.1) 

0 

(1.1) 

1 

(1.3) 

3 

(1.2) 

(1,2) 

1 

0 

(1.3) 

2 

(1,3) 

(1.2) 

2 

(1.3) 

1 

(1.3) 

1 

(1.4) 

(1.2) 

3 

(1,3) 

2 

0 

(1.5) 

(1,2) 

4 

(1.3) 

3 

(1.5), 

1 

(1.6) 

(1.2) 

5 

(1,3) 

4 

(1.5) 

2 

(1.7) 

(1,2) 

4 

(1,3) 

3 

(1,7) 

1 

(1.  ) 

(2,  ) 

00 

00 

00 

at  node 

0 

,D 

(1,2) 

(1,4) 

Table  3. 2. a RT's  at  the  Exchange  Nodes 


Destination 


i 


I 

1 


£ 


(2,1) 

J . < f 

0 

(2,1) 

1 

(2.1) 

3 

(2,2) 

(2,2) 

1 

(2,1) 

2 

(2,6) 

4 

(2,3) 

(2,2) 

2 

(2,4) 

2 

(2,6) 

4 

(2,4) 

(2,5) 

2 

(2,4) 

1 

(2,6) 

3 

(2,5) 

(2,5) 

1 

0 

(2.6) 

2 

(2,6) 

(2,5) 

2 

(2.6) 

1 

(2,6) 

1 

(2,7) 

(2,5) 

3 

(2,6) 

2 

0 

(1,  ) 

00 

00 

00 

(2,  ) 

* 

at  node 

(2,1) 

( 

2,5) 

( 

2,7) 

Table  3.2.b  RT's  at  the  Exchange  Nodes 


Let  us  now  look  at  the  exchange  of  updates  between  nodes  in 
different  clusters.  As  an  example,  assume  that  node  (2,5)  receives 
update  from  node  (1,1).  The  first  7 entries  (in  their  RT‘s)  do  not 
refer  to  common  destinations  and  as  such  they  are  only  utilized  to 
coapute  the  aggregate  routing  information.  That  information  (0  for  CER, 
average  for  OBR)  is  counted  and  stored  in  the  1st  level  cluster  self 
entry  at  node  (1,1) 's  RT,  by  either  the  sending  or  receiving  node. 
(Results  of  such  computations  are  shown  in  Table  3.3.) 


, „ , 
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1 

Content 

1 

p ♦ 5/7 

2 

! 1 +3/7 

of  entry  , 
(1.  ) 

0 

0 

1 0 

1 

at  node 

(1,1) 

(1,2) 

(1.4) 

Table  3.3  Aggregation  of  Routing  Information 


OB)l 

CER 


After  the  computation  of  the  aggregate  information.  Algorithm 
3.1  is  performed.  The  outcome  of  this  single  step  is  shown  in  Table 
3.4  for  all  the  exchange  nodes  in  Cj(2). 


Entry 

(1.  ) 

at  node 


(1,4) 

2 ♦ 3/7 

(l.D 

3 + 5/7 

(1,2) 

3 

(1,4) 

1 

(l.D 

1 

(1,2) 

1 

(2,1)  (2,5)  (2,7) 


OBR 


CER 


Table  3.4  Intermediate  Update 


This  information  recorded  in  entry  Cj(l)  in  the  above  RT's  will 
percolate  inside  cluster  Cj(2).  At  one  point  (2,5)  will  receive  an 
iq>date  from  (2,1)  which  will  (if  OBR  is  used)  trigger  a change  in 
Cj(l)'s  entry.  It  can  be  easily  checked  that  entries  reserved  to  C^(l) 
in  (2,1)  and  (2,7)  will  not  be  affected  by  updates  coming  from  nodes  in 
Cj(2).  As  a result,  the  final  outcome  for  such  entries  is  as  shown  in 


Table  3.5. 


Table  3.5  Final  Outcome 


Notice  that  the  resulting  average  path  length  from  (2,5)  to  the  nodes 
in  C.(l),  is 


3 + 5/7  CER  (computed  from  Table  3.5) 

3 + 3/7  OBR  (see  Table  3.5) 

2 ♦ 4/7  NCR  (see  Table  3.1) 


As  for  (2,1)  and  (2.7),  that  average  path  length  remains  the  same  with 
either  one  of  the  hierarchical  schemes.  The  comparison  of  the  above 
numbers  shows,  as  expected,  that  globally  CER  will  induce  a longer  net 
work  path  length  than  OBR  which,  in  turn,  induces  a longer  path  than 


Routing  Function 


The  problem  here  is  to  decide  upon  the  RT  entry  to  utilize  in 
order  to  forward  (or  to  send)  a message  from  a node  of  address 
i * (im’  **-1'  •••»  ij)  to  a node  of  address  j * (j^,  ...,  jp 

(see  Section  2.3. 1.1).  Let  'as  first  notice  that  if  k is  the  level  of 
the  lowest  level  cluster  to  which  both  nodes  belong,  then  their  address 
vectors  mist  match  starting  from  the  left,  up  to  and  including,  the 
index  k ♦ 1,  i.e.. 


ip»jp  p ■ m,  m - 1,  . . . , k + 1 

‘k  * ik 

As  an  example  we  compare  the  addresses  of  a few  pairs  of  nodes  chosen 
from  the  3 level  clustered  network  shown  in  Fig.  2.1.  The  outcome  of 
the  comparison  is  given  in  the  table  below. 


(ij.iziij) 

(i3*  j2»  jp 

k 

(1,3,3) 

(3,2,1) 

3 

(1,3,3) 

(1.2,2) 

2 

(1,3,3) 

(1.3,2) 

1 

Table  3.6  Matching  the  Address  Vectors 


Because  of  the  above  property,  the  routing  algorithm  can  then 

easily  perform  a sequential,  from  left  to  right,  matching  of  the  address 

vectors  to  find  the  value  of  k.  Then  the  next  node  field  of  the  i, th 

Jk 

St 

entry  among  the  k-1  level  cluster  entries  (see  Fig.  3.1)  will  indicate 
the  next  node  on  which  to  forward  the  message. 

With  the  above  specifications  of  the  MIR  and  NCR  schemes,  we 
are  now  ready  to  address  the  question  as  to  what  is  the  content  of  the 
hop  fields  at  any  RT,  under  some  defined  equilibrium  conditions. 

3.1.3  Path  Characteristics 

If  no  changes  occur  in  the  topology  of  the  network,  after  a 
certain  number  of  updates  the  contents  of  the  hop  fields  in  the  routing 
table  will  reach  "minimal”  constant  values.  In  what  follows,  this 
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situtation  will  be  referred  to  as  equilibrium  condition.  Similar  to 
the  dynamic  programming  approach,  the  above  property  is  due  to  the  fact 
that  improvements  are  made  sequentially  at  each  ipdate  over  the  distance 
from  one  node  to  any  cluster  (see  Algorithm  3.1).  The  question  arises 

I 

as  to  what  is  the  meaning  of  the  routing  information  at  equilibrium,  or 
in  other  words,  what  are  the  characteristics  of  the  paths  indicated  by 
the  routing  tables.  We  have  already  noticed  that  for  the  degenerate 
one  level  hierarchical  clustering,  i.e.,  when  no  clustering  is  used, 
those  paths  correspond  to  the  shortest  paths  in  the  current  topology. 

Before  we  proceed,  a few  more  definitions  and  notation  are  necessazy. 


hgt  = Length  of  the  estimated  minimum  path  from  node  p to 
node  t as  derived  from  the  routing  information  at 
node  s.  (The  superscript  c stands  for  clustered 
routing. ) 


Internal  path  * a path  is  defined  to  be  internal  (included)  in  a 

cluster  if  all  the  nodes  in  that  path  belong  to 
that  cluster. 

h^t  * Length  of  the  shortest  path  from  node  s to  node  t 
included  in  the  lowest  level  cluster  to  which  both 
s and  t belong. 

i 

Exchange  node  - (defined  previously)  An  exchange  node  (to  be  denoted 

* 

by  e or  e^)  of  a given  cluster  is  a node  of  that 
cluster  which  is  connected  to  one  or  more  nodes 
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A^Ci^j)  * Subset  of  all  the  exchange  nodes  which  connect 

cluster  with  any  k*1*  level  cluster  which 

belongs  to  the  sane  k+lst  level  cluster  as  C^Ci^). 

w » Aggregate  variable  representing  cluster  C.  and  as 
eck  * 

confuted  from  the  routing  information  contained  at 

the  exchange  node  e of  C^, 

From  the  above  definitions  and  previous  specifications  we  no- 
tice first  that  a network  node  (0th  level  cluster)  is  its  own  exchange 
node,  second  that 


— £ 


for  the  OBR  scheme 


for  the  CER  schei 


(3.2) 


where  | | represents  the  number  of  nodes  in  cluster  and  f is  an 

arbitrary  node  ofC^.  The  above  considerations  allow  us  to  character- 
ize the  path  lengths  under  the  MIR  schemes. 

Proposition  5.1 

Let  s and  t be  two  arbitrary  nodes  which  belong  to  the  same  k^ 
level  cluster  but  not  to  any  lower  level  cluster;  then  the  length  of 
the  path  from  node  s to  node  t as  derived  at  equilibrium  from  the  routing 
information  contained  at  node  s,  satisfies  the  recursive  equation  below. 


h»o  * hv 


(3.3) 
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■ «jwr^-flq«nr' 


where  eQ  is  an  exchange  node  of  C^jCt)  which  is  such  that 

h*e0  * WeOCk-l(t)  “ •jeJ^1(t){hsej  * *#jCk-l(t)j  <3*4) 


where  C^Ct)  is  the  k-lst  level  cluster  which  contains  node  t,  and 

is  its  corresponding  subset  of  exchange  nodes  as  defined  above. 


Proof 


The  proof  proceeds  by  induction  on  the  level  k of  the  lowest 


level  coanon  cluster.  In  what  follows  ^(s)  and  C^(t),  ) - 1,  ....  «, 

,th 


will  always  respectively  denote  the  j level  clusters  to  which  s and  t 
belong. 
k « 1 

s,  t belong  to  the  si 

C0(s)  - AqCs)  - {s} 

C0(t)  - Ap(t)  - (t) 


1 level  cluster  Cj,  then 


Also,  since  the  distance  of  a node  to  itself  is  zero,  then 

0 


hv ' h« 


In  order  to  prove  Eq.  (3.3)  there  renains  to  show  that 


st  nst 

i.e.  * t^iat  is  the  length  of  the  shortest  path  fron  s to  t included 
in  Cj.  This  is  true  since  the  RT  of  any  node  in  Cj  contains  an  entry 
for  node  t;  hence  at  equilibrium  we  obtain  the  minimal  internal  path 
from  s to  t.  Notice  that  if  n « 1,  i.e.,  the  degenerate  case,  all  nodes 
belong  to  the  same  cluster  Cj  which  corresponds  to  the  entire  set  of 
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1 4 

: i 


In  other  words , when  no  clustering  is  used 


nodes,  hence  h 


the  routing  information  indicates,  at  equilibrium,  the  shortest  (hop) 
path;  which  checks  with  the  conclusion  reached  in  Section  3.1.1. 

Assuming  that  Proposition  3.1  is  true  up  to  k - 1,  let  us  show 


Proof  for  k 


be  the  k level  cluster  common  to  s and  t.  All  the 


nodes  in  contain  in  their  RT's  one  entry  for  cluster  (^^(t).  Th* 
propagation  and  the  subsequent  updating  of  the  RT's  among  the  nodes  of 
C^,  is  equivalent  to  finding  the  minimum  path,  internal  to  C^,  from  any 
node  in  {C^  - (^^(t)}  to  the  fictitious  supemode  SC^  jCt)  shown  in 

seen  from  any  node  in  {c^  - CyjCt)}, 


In  other  words 


Figure  3.3.  Equivalent  Representation  of  Clutter  C.  . (t) 


cluster  is  equivalent  , in  terms  of  distance,  to  a center  node 

SCjt_j(t)  connected  to  all  the  exchange  nodes  in  A^  j(t).  If 

e.  e A^_j(t)  then  the  length  of  the  equivalent  edge,  from  e.  to  the 


center  node,  is  equal  to  the  aggregate  information  representing  cluster 
Ck_1 (t)  as  seen  from  e^,  i.e.. 


i(e 


'r  sck-i(t»  ■ Vc^M 


(3.5) 


Notice  if  the  CER  scheme  is  used  then  the  above  equivalent  represen- 
tation reduces  to  the  single  node  SC^ft).  In  general,  from  Eq.  (3.2) 


e. 


l^-i^l  ffe^ct)  Kejf 


(OBR) 


(3.6) 


(CER) 
c 


Since  e.  and  f belong  to  the  same  k-lst  level  cluster  then  hc  , is 
J e . f 

J 

defined  because  of  the  induction  hypothesis.  Hence,  the  above  equation 
is  known. 


If  eQ  is  the  exchange  node  in  Afc_1 (t)  which  belongs  to  the 
minimal  path  from  s to  SCj{_j(t)  obtained  at  equilibrium,  then  e^ 
satisfies  Eq.  (3.4)  which  represents  the  length  of  that  minimal  path. 
Due  to  the  routing  function,  previously  specified,  all  messages  to  be 
forwarded  or  sent  from  node  s to  node  t will  follow  the  same  minimal 
path  up  to  the  exchange  node  eQ.  At  that  point  eQ  and  t belong  to  the 
same  k-1  level  cluster,  hence,  h*"  is  known  from  the  induction 

V 

hypothesis.  Consequently 


st 


se„ 


♦ h 


V 


where  eQ  satisfies  Eq.  (3.4). 


Q.E.D. 
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Remarks 


If  the  CER  scheme  is  used,  then  Eq.  (3.4)  becomes 


min  {h*  } 

- a 


(3.7) 


0 j 

The  above  equation  indicates  that  eQ  is  the  closest  exchange  node  of 
Ck_1(t)  (for  paths  included  in  )to  node  s.  This  explains  the 

nomenclature:  Closest  Entry  Routing. 

ii*  Arbitrary  Channel  Lengths.  Instead  of  assigning  (as 
above)  length  1 (1  hop)  to  all  the  channels,  we  intend  to  let  the  chan- 
nels be  of  variable  lengths.  Let  a^  represent  the  length  of  channel 
(i,j),  then  the  length  of  a path,  tt  , from  node  s to  node  t is  defined 


i^st)  » E 


(3.8) 


The  specifications  of  the  MIR  and  the  NCR  schemes  are  still  valid 
except  that  any  hop  distance  is  replaced  by  the  new  assigned  length. 
Hence  Algorithm  3.1  becomes 


IF  HF(t,  Q.(i))  > ats  + HF(s,  C..(i)) 

WEN  HF(t,  Cj(i))  <-  ats  ♦ HF(s,  C.(i)) 

NEXT  NODE  FIELD  OF  C..  (i)  «-  s 


(3.9) 


0 X 

Also  if  we  let  hjt,  hgt,  and  wec^  t>e  88  defined  previously,  then 


Proposition  3.1  still  holds  true. 


I 


I 


3.2 


Bounds  on  the  Increase  in  Path  Length 

The  effect  of  the  clustering  (reduction  of  routing  information) 
is  the  increase  of  the  path  length  between  any  pair  of  nodes,  s,  t, 
of  an  amount  hgt  - h^.  A measure  of  performance  of  the  MIR  schemes 
could  be  the  relative  increase  of  the  average  path  length,  i.e., 

h 

D = jp  - 1 (3.10) 

where  hc  and  h denote  the  average  path  length  in  the  network  respec- 
tively with  and  without  clustering, 

h ' srnr  ££  hst 

(3.11) 

”c  ‘ Nw’-  D h» 

Proposition  3.1  provides  a means  for  computing  the  values  of 
c 

for  any  pair  of  nodes  s,  t,  for  a given  outcome  of  the  m-level 
hierarchical  clustering  of  the  set  of  nodes,  S.  Consequently,  for 
that  particular  situation,  it  is  numerically  possible  to  evaluate  the 
relative  increase  D,  Eq.  (3.10)  and  then  compare  the  clustered  with  the 
non- clustered  schemes.  Moreover,  with  further  assumptions  on  the 
structure  of  the  hierarchical  partitioning  of  the  nodes,  we  can  obtain 
analytic  bounds  on  the  increase  in  the  path  length. 

Assumption  3.2 

The  diameter1  of  any  level  cluster  subnet  (see  assunqttion 
3.1)  is  less  than  or  equal  to  a quantity  d^,  k = 1,  ...,  m. 

1Recall  that  the  diameter  of  a network  is  the  maximum  shortest  path 
between  pairs  of  nodes  [HARA  72]. 
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Notice  that  d^  represents  the  diameter  of  the  entire  network 
and  that  d^  > d^  ^ _>  0 for  all  k*s. 

Assumption  3.3 

Any  cluster  at  any  level  k = 1,  2,  ....  m contains  the 
shortest  path  (if  it  is  not  unique,  then  at  least  one  is  contained) 
between  any  given  pair  of  nodes  which  belong  to  that  cluster. 

Assumption  3.2  is  simply  the  specification  of  the  outcome  of 
the  clustering  of  the  nodes,  since  the  d^*s  can  be  of  any  value. 

Whereas  Assumption  3.3  is  a natural  property  that  any  clustering  scheme 
should  seek.  The  reason  for  this  is  that  traffic  between  nodes  in  the 
same  cluster  must  (because  of  the  routing  function  above)  follow  paths 
internal  to  that  cluster. 

The  above  assumptions  lead  to  the  derivation  of  some  single 
bounds.  These  bounds,  on  the  increase  in  path  length,  pertain  to  the 
routing  schemes  (OBR,  CER)  described  in  Section  3.1.  All  the  prop- 
erties listed  below  rely  on  Assumptions  3.1  and  3.2.  If  Assumption  3.3 
is  used,  it  will  be  so  specified. 

Lemma  3.1 

Uhder  the  above  conditions,  the  value  of  h t for  any  pair  of 

nodes  s,  t which  belong  to  the  same  k**1  level  cluster  is  such  that 

■V  s,  t e same  k level  cluster 
d.  (3.12) 

J V-  k » 1,  2,  . . . , m 

Proof 

The  proof  proceeds  by  induction  on  k.  First  if  k ■ 1 then 
similar  to  the  proof  of  Proposition  3.1  (for  k = 1)  h^t  * h^.  Then 
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Now  assuaing  that  Leona  3.1  is  true  for  all  levels  up  to  k - 1,  let  us 
show  that  it  is  true  for  k. 

Let  s,  t be  a pair  of  nodes  which  belong  to  the  same  kth  level 
cluster  C^.  Let  be  the  lowest  level  cluster  to  which  both  s and  t 
belong,  hence  p £ k.  If  p < k then  from  the  induction  hypothesis. 


K3 
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Proof 


Let  Vs>  denote  the  fcth  level  cluster  (k  = 0,  1,  ....  m)  to 
which  s belongs.  Then  from  Eq.  (3.11), 


h - h 
c 


N(N  - 1)  £ 51  !£  thst  " hst} 

1 } seS  k*l  teCj^Cs)  st  st 


(3.14) 


tiCk.ifs) 


The  above  egression  was  obtained  through  the  decomposition  of  the 
computation  of  the  average  over  all  the  nodes  in  the  network.  Then  for 
a given  node,  s,  the  decomposition  is  done  over  nodes  in  the  same  1st 
level  cluster  as  s,  excluding  s (CQ(s)  - s),  then  over  nodes  in  the  same 
2 level  cluster  as  s,  excluding  those  in  the  same  1st  level  cluster 
as  s,  etc... 

s t 

C^O)  be  a k-1  level  cluster  included  in  C^(s);  there 
are  n^  such  clusters,  then 

«$•>  th“  ' V = I.  «fl0,  (h«  ' h*) 

. (3.15) 

Since  n (^^(s)  = <{>  and  both  are  included  in  (^'s,  Eq.  (3.3) 

holds  true  for  s and  any  node  t in  (j),  hence 

«^o)  h”o  * «£(,,  "v  (3-16) 

where  e^  satisfies  Eq.  (3.4).  Two  cases  to  consider  are 
i.  OBR  scheme.  From  Eq.  (3.2) 
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(3.17) 


Vk-1«>  = IViH'l  ftc^cj)  h'of 


Substituting  Eqs.  (3.17)  and  (3.4)  (in  Eq.  (3.4)  t is  replaced  by  j) 
into  Eq.  (3.16),  we  arrive  at 


£ hst  * min  {hL  + w r ril 

teCk-im  ^ 1 e.eA^j)  sej  Vk-iO)' 


(3.18) 


Let  us  define  e^  to  be  the  closest  (inside  C^(s))  exchange  node  of 
AJc_i ( j)  to  node  s,  i.e.. 


se 


s j 


(3.19) 


From  Eq.  (3.18)  and  for  any  exchange  node  e^,  particularly  es,  the 
relation  below  is  true. 


tec£«)  h-  2 * «£0)  hV  (5-20) 


Note  that  in  the  equation  above  w was  replaced  by  its  value  as  defined 
by  Eq.  (3.2). 

Moreover,  from  Assunption  3.3 


h = h . 
st  st 


Vs,  t 


(3.21) 


Then  from  the  definition  of  e , 

s’ 


h = h*  > h* 
st  st  — se 


V t 


(3.22) 


Consequently, 
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I ft 


• « 


1 


i 


iWa^a'i  tm , ini  i iaiii  ftilrarrnT 


(3.23) 


£m  hstiick-i«)i  hi 


se 


Substituting  Eq.  (3.23)  into  Eq.  (3.20),  we  arrive  at 


I 0>,t  - l>  ) < £ hf 

teCk-i(j)  teCk-l(j)  5 


Note  that  es>  t e C^_j(j),  then  from  Lemma  3.1, 

r k“! 

he  il  di  * t e C.  ,(j) 
est  j=l  3 k 1 


From  Assumption  3.1 


|^_j(j)|  = ’ * ***k-l  j 


(3.24) 


(3.25) 


(3.26) 


Substituting  Eq.  (3.24  - 3.26)  into  Eq.  (3.15),  we  find 

k-1 

i-  (hst  - *»  ) 1 (nk  - l)n.n....n  . £ d.  (3.27) 

teC  (s)  st  st  k l / k-l  j=1  j 

**<vi  (s> 


Note  that  this  last  equation  is  true  for  any  level  k,  and  for  any  node 
s,  hence  by  substituting  it  into  Eq.  (3.14),  we  obtain 

m k-l 


h - h 
c 


2 1,1  IVA 

— n - 1 nln2‘ ' "nk-l^nk  " ^ 

k=l  * jal  J 


Note  that  for  k = 1, £ =0. 


Interchanging  the  summations  in  the  equation  above,  we  get 

m-1  m 

L d5  L 

j«=l  J kaj  + l 


j m-i  m 

hc  - hirn  £ dj  ”iV--Vi(nk  - ’> 
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which  after  some  algebra,  gives  Eq.  (3.13). 

To  end  the  proof  it  is  left  to  show  that  the  CER  scheme  also 
satisfies  Proposition  3.2. 


ii.  CER  Scheme.  From  Eqs.  (3.2)  - (3.4)  and  (3.19),  we  find 


st 


h1  + hc 

ses  V 


teck-i'» 


hsV  * 


e t 


teC^j  (j)  s 


(3.28) 


This  equation  is  equivalent  to  Eq.  (3.20)  except  that  the  inequality  is 
tight  here.  Two  conclusions:  (1)  the  rest  of  the  proof  can  proceed 
exactly  as  in  (i),  (2),  Eq.  (3.28)  coup ared  to  Eq.  (3.20),  indicates 
that  the  summation  of  path  lengths  obtained  with  the  OBR  scheme  is 
smaller  than  or  equal  to  the  one  obtained  with  the  CTR  scheme.  Hence 
th®  average  path  with  an  OBR  is  smaller  than  or  equal  to  the  average 


path  length  with  a CER. 

The  above  proposition  deals  with  averages;  we  now  intend  to  put 
a bound  on  the  increase  of  the  path  length  between  an  arbitrary  pair 
of  nodes  s,  t. 


Lemma  3.2 


Under  the  previous  conditions  and  Assumption  3.3,  and  for  the 
CER  scheme 


k-1 


th 


st 


h t < y d. 
st-j=i  ^ 


¥ s,  t e same  k level  cluster  C^ 


V-k  = 1,  2,  ...,  m 


(3.29) 
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Proof 


Under  the  above  condition  Eq.  (3.28)  holds  true 


h«,  » h*  ♦ hc 
st  se  e t 
s s 


Since  es  , t e * then  from  Lemma  3.1 


_ k-1 

V i-1  1 


Also,  because  of  Assumption  3.3,  and  Eq.  (3.19) 

h 


. i h\  > h1 
st  st  — se 


Oufcining  the  last  three  relations  together,  we  find  Eq.  (3.29). 

Note  that  lemma  3.2  is  not  true  with  the  OBR  scheme,  because 


such  a scheme  tries  to  find  the  best  overall  exchange  node  (eQ)  to 


route  the  messages  to,  instead  of  using  the  closest  one. 

We  observed  previously  that  Assumption  3.3  is  a realistic  one, 
but  if  it  is  not  specifically  built  into  the  clustering  algorithm,  there 
is  no  guarantee  that  the  outcome  of  the  clustering  always  satisfies  that 
assunqption . This  remark  leads  us  to  the  following  proposition. 


Proposition  3.3 

Under  the  conditions  of  Proposition  3.2  and  with  Assumption  3.3 

removed. 


m-1 

hc  - h i “k 


k-1 


(3.30) 


Proof 


Let  s,  t be  an  arbitrary  pair  of  nodes  and  let  be  the  lowest 


i ! 


II 


i 1 


- 
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level  cluster  to  which  s,  t belong.  TVo  cases  to  consider  are 
i.  k < m.  From  Lemma  3.1 

k 


Hence 


h < V d. 

)=1  ‘ ]=1  ‘ 


m-1 


ii.  k = m.  Cm  represents  the  entire  set  of  nodes,  hence 


h1.  = h . 
st  st 


Then  similar  to  the  proof  of  Proposition  3.2,  we  can  show  that  Eq. 
(3.27)  holds  true  for  k ■ m,  i.e., 

m-1 

£ Chst  " hst)  * (nm  ' 1)nln2”*Vl  £ di  (3*32) 

teC  (s)  51  sz  m i*l  J 

m ' 

'K-i^ 


Because  of  Eq.  (3.31)  and  Assumption  3.1 


m-1 


£ ...  (hst  - hst>  S dj 


(3.33) 


t£Cm-l(s) 


Substituting  Eqs.  (3.32)  and  (3.33)  into  (3.14)  we  arrive  at  Eq.  (3.30). 
Conclusion 

In  summary,  several  fairly  general  bounds  have  been  derived, 
depending  on  the  assunptions  and/or  the  routing  schemes  selected.  In 
the  next  paragraph  we  will  study  the  behavior  of  some  of  those  bounds 
in  the  context  of  a defined  class  of  networks.  Let  us  also  show  that 
for  the  degenerate  case  of  1-level  hierarchical  routing  (NCR  scheme) 
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the  bounds  derived  over  the  increase  in  path  length  are  tight;  i.e., 
for  m » 1,  Eqs.  (3.13)  and  (3.30)  become 

h - h < 0 
c — 

but  since  h - h > 0 ->  h = h.  Also  Eq.  (3.29)  becomes  hc.  a h .. 

c c St  St 

3. 3 Static  Performance  Evaluation  of  the  MiR  Schemes  for  a Family 
of  Networks 

If  tenporarily  we  do  not  take  into  account  the  significant 
gains  obtained  in  reducing  the  CPU,  storage  and  line  utilizations 
required  by  the  routing  procedures,  then  the  application  of  the  MHR 
schemes  will  result  in  a degradation  of  the  performance  of  the  network, 
as  compared  to  the  utilization  of  a non-clustered  scheme.  The  loss  in 
performance  (delay,  throughput)  is  closely  related  to  the  average  path 
length  a message' follows  in  the  network.  The  evaluation  of  the  increase 
in  path  length  provides  us  with  a first  cut  modeling  of  the  loss  in 
network  performance.  Moreover,  the  study  of  the  bounds,  derived  pre- 
viously, represents  a worst  case  evaluation  of  the  (HR  schemes.  Since 
the  evaluation  is  in  terms  of  path  length,  we  will  refer  to  it  as 
static  performance  evaluation.  The  gains  obtained  are  modeled  by  the 
single  variable  £./N  which  represents  the  reduction  of  routing  infor- 
mation (we  will  refer  to  Jl/N  as  the  relative  table  length).  The  static 
performance  evaluation  is  performed  over  a class  of  computer  networks. 

3.3.1  A Family  of  Large  Distributed  Networks 

The  networks  to  be  considered  are  all  the  connected  graphs 


upon  which  it  is  possible  to  fit  an  m-level  hierarchical  clustering 
whose  outcome  satisfies  Assumptions  3.1  - 3.3.  Also  the  resulting 
cluster  subnets  at  any  level  are  of  diameters  bounded  by  a power  law 
function  of  the  number  of  nodes  in  that  cluster;  i.e.,  if  n is  the  size 
of  a cluster  and  d the  diameter  of  that  cluster's  subnet  then 

d < bnV  ♦ c (3.34) 


where  b,  c,  v are  positive  parameters  and  0 £ v £ 1 (see  below). 

If  N is  the  size  of  such  a network,  then  the  average  path 
length  (hop  distance)  of  that  network,  h,  must  be  a power  law  function 
of  N, 

h » aNV  (3.35) 

where  a is  a positive  parameter. 

Grid  type  networks  [BARA  64],  hexagonal  networks,  etc.,  fall 
into  that  category  when  the  IflC  results  in  subnetworks  of  similar  struc- 
ture as  the  original  and  when  the  path  lengths  are  expressed  in  hops. 
Expressions  for  the  average  path  length  (with  a uniform  traffic  matrix 
Y^j  * y)  and  for  the  diameter  of  the  grid  and  the  torus  networks  have 
been  derived  in  Appendix  A.  Some  of  the  results  obtained  are: 


Square 'grid  of  size  N 


is* 


Square  torus  of  size  N, 
(with  /N  odd) 


d =*  2 /7T-  2 

h = -j 

d - yir  - 1 


(3.36) 


(3.37) 
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Furthermore,  if  the  partitioning  of  either  the  square  grid  or 
torus  networks  results  in  grid  cluster  subnets  at  all  levels  (see  Fig. 
5.6),  then  for  any  cluster  subnet  of  size  n its  diameter  d is  such  that 

d 4 2 /n  ~ 2 (3. 38) 

As  a consequence  the  grid  and  torus  networks  fit  the  above  descriptions. 
Note  also  that  for  those  networks  the  exponent  v (Eqs.  (3.34)  and 
(3.35))  is  equal  to  1/2. 

In  general,  the  exponent  v reflects  the  connectivity  of  the 
network  considered.  For  very  highly  connected  networks  v is  in  the 
neighborhood  of  zero;  e.g.,  for  a fully  connected  network  v = 0 
(h  « 1,  d * 1).  Whereas  for  very  low  connected  networks  v is  in  the 
neighborhood  of  one;  e.g.,  for  loop  or  chain  type  networks,  v = 1. 

Computer  communication  networks  fall  into  the  class  of  dis- 
tributed networks.  This  class  includes  networks  such  as  the  ARPANET, 
the  NPL  network  [DAVI  73),  the  Cyclades  network  [ZIMM  75],  TELENET, 
etc.  The  main  characteristic  of  those  distributed  networks  is  their 
low  connectivity.  In  general  a connectivity  2 (or  3)  is  imposed  on 
their  design.  For  large  distributed  networks  a connectivity  of  3 to  4 
seems  more  appropriate  ( NAC  73).  The  torus  networks  considered  above 
are  of  connectivity  4 and  with  an  exponent  v = 1/2,  hence  they  appear 
to  be  good  representatives  of  large  distributed  networks.  Moreover, 
their  topological  structures  lead  to  simple  partitions  such  as  square 
subgrid  clusters.  In  the  sequel,  we  will  first  derive  a limiting 
result  valid  for  the  entire  class  of  networks,  then  we  will  restrict 
our  considerations  to  values  of  a,  b,  c,  v as  obtained  for  a torus. 
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3.3.2  Limiting  Performance  of  the  MiR  Schemes 


As  mentioned  earlier,  the  static  performance  evaluation  of  the 

i 

MiR  schemes  involves  the  determination  of  a bound  on  the  relative 
increase  in  the  average  path  length.  For  the  above  family  of  networks, 
the  size  of  a kth  level  cluster  is  njn2...nk  and  because  of  Eq.  (3.34), 

d^  = bfnj^. . .nk)V  + c k = 1,  2,  ...»  m (3.39) 


This  family  of  networks  also  satisfies  Assumptions  3.1  - 3.3,  hence 
Proposition  3.2  holds  true.  From  Eqs.  (3.10),  (3.13)  and  (3.39) 

. v 


D = ir- 1 ±E  = 


a(N  - 1)NV  k’ 


t (N  • A n^A  ■*)  * j 


(3.40) 

Notice  that  E is  defined  as  the  bound  on  D.  It  is  the  behavior  of 
E versus  i/N  that  we  are  interested  in.  TVo  cases  will  be  distinguished 
in  the  study  of  E;  first  when  the  MiC  results  in  a minimum  table  length, 
Eq.  (2.6),  second  when  the  MIC  results  in  a vector  minimizing  the  bound 
E for  a given  table  length  i. 

For  an  optimal  clustering  structure  we  know  from  Proposition 
2.1  that  the  degree  vector  must  satisfy  Eq.  (2.5).  Substituting  Eq. 
(2.5)  into  Eq.  (3.40),  we  arrive  at 


a(N  - 1)N  k 


*1  [n  - Nk/*][bNkv/"  . c] 


and  if  we  restrict  v to  be  different  from  zero,  then  after  some 
algebra 


H 

B'  j 

li 


I 


1 


p^: 


I 


a(N  - 1)n' 


I Nv  - NV/^m  1 mv+1  m 

T^r*c(m-1,J  ■hlsrl 


v+1 
V+1  „ m 


- c 


N - N 


1/m 


N m - 1 


N1/m  - 1 


(3.41) 


Notice  again  that  for  m = 1,  E = 0. 

Also  from  Eq.  (2.6),  the  relative  table  length  is 

l _ mN1/m 


(3.42) 


N N 

The  above  considerations  lead  to  the  general  limiting  result  below. 


Proposition  3.4:  Limiting  Performance 

Consider  the  above  family  of  networks  and  the  above  MR  schemes 
(OBR,  CER)  with  a fixed  number  of  levels  m and  an  optimal  clustering 
structure.  Then,  as  N,  the  number  of  nodes,  goes  to  infinity,  the 
"static"  performance  of  the  MIR  schemes  approaches  that  of  a non- 
clustered routing  scheme,  while  the  relative  table  length  approaches 
zero;  i.e.. 


hc 

IT*  1 


N ■+•  OO  =.> 


(3.43) 


Thus  we  claim  we  can  get  the  best  of  both  possible  worlds. 


Proof: 


From  Eq.  (3.42),  the  above  limit  of  &/N  is  obvious  (recall 
that  m is  fixed).  Also  from  Eq.  (3.3),  it  is  obvious  that 


hst  i hs« 


¥ s,  t e S 
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hence 


h > h > 0 . 
c — — 


Then  because  of  Eq.  (3.40) 


0 < jp  - 1 £ E 


(3.44) 


As  a consequence,  it  is  enough  to  prove  that  the  limit  of  E is  zero. 
Expanding  Eq.  (3.41)  around  N \ we  find 


E . | fTv/l"  . o(n'v/i") 


(3.45) 


hence 

limit  E = 0 Q.E.D. 

N-*» 

Notice  that  the  closer  v is  to  one  (v  ^ 0) , the  faster  is  the 
convergence  of  E to  zero.  In  other  words,  as  could  be  expected,  the 
more  distributed  (the  less  connected)  the  networks  are  the  better  the 
MUR' s perform.  Notice  also  the  enormous  gains  in  table  length  for  a 
relatively  insignificant  increase  in  the  average  path  length. 

r 

The  above  results  hold  true  if  we  relax  Assumption  3.3  imposed 
over  the  outcome  of  the  clustering  of  any  of  the  networks  which  belong 
to  the  family  considered  here.  To  prove  this  statement  let  F be  the 
bound  on  the  relative  increase  in  path,  as  obtained  from  Proposition 
3.3,  i.e.,  from  Eq.  (3.30)  and  (3.39), 

h . m- 1 

- 1 < FS  -L-  £ [b(n.n  ...n  )v  ♦ c]  (3.46) 

n ” aNv  k=l  14k 

» 

For  an  optimal  clustering,  the  above  equation  becomes. 
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F * 


aN 


' Nv  . Nv/m 
” N^"-  1 ‘ C‘"  ' ” 


(3.47) 


Similar  to  E,  as  N goes  to  infinity,  F goes  to  zero,  which  implies  that 

h /h  goes  to  1. 
c 

The  result  of  Proposition  3.4  was  derived  for  a fixed  m;  let 
us  now  examine  the  situation  where  m is  variable.  Of  interest  is  the 
value  of  m which  corresponds  to  the  global  optimum  clustering  structure. 
That  value  is,  from  Eq.  (2.17),  m*  = lnN. 

Substituting  Eq.  (2.17)  into  Eq.  (3.41),  we  arrive  at 


E*  - E 


1 


a(N  - 1)N' 


Nv  - Av 

bN  — — — + c(ln  N 
e - 1 


- 1)N 


N1*V-  e1^ 
1+v 


N - e 
e - 1 


(3.48) 


Then 


limit  E = - 
N-  * a 


ev  - 1 e1+v-  1 


(3.49) 


As  a consequence  the  result  of  Proposition  3.4  is  not  necessarily  true 
anymore.  If  we  consider  gTid  or  torus  networks,  then  from  Eqs.  (3.36) 
(3.38), 

a = -|,b=2,  c = -2,  v = j 

! ! (3. SO) 

a * j , b » 2,  c = -2,  v = j 

Substituting  the  above  values  into  Eq.  (3.49),  we  arrive  at 


| for  a grid: 
( for  a torus: 
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i 5.01  for  torus  nets 

limit  H*  = j (3.51) 

N-*00  ( 3.  76  for  grid  nets 


Fig.  3.4  illustrates  the  behavior  of  E*  (for  torus)  with  respect  to  N. 
The  curve  shows  E*  as  an  increasing  function  of  N.  It  also  shows  that 
the  cost  of  operating  at  the  (global)  minimum  table  length  may  become 
quite  high  (up  to  6 times  increase  in  path  length) . Fortunately,  as 
noticed  in  Chapter  2,  most  of  the  table  reduction,  for  practical  pur- 
poses, may  be  obtained  with  m quite  a bit  smaller  than  the  global 
number  of  levels  m*,  and  as  we  will  see  in  later  plots  (Figs.  (3.5)  and 
(3.7))  the  cost  at  a small  m is  quite  minimal.  Those  figures  also  show 

l • , — — . - _ ;]  r,  ^ 

the  behavior  of  E*  versus  £/N. 


3.3.3  Static  Performance  Evaluation  for  the  MIR's:  Numerical 


In  the  previous  section  we  observed  that  at  the  limit  (N  -*■  «) 


considerable  table  reduction  can  be  achieved  with  no  loss  in  performance. 
In  this  paragraph,  we  intend  to  look  at  the  more  general  case  of  a finite 
N.  The  purpose  is  to  correlate  the  degradation  in  performance  with  the 
table  reduction.  More  precisely,  we  will  evaluate  a maximum  performance 
degradation  in  teims  of  the  gains  in  table  length.  Also  this  evaluation 
will  be  carried  out  with  an  HiC  which  results  first  in  a minimal  table 


length  and  second  in  a minimal  bound  E 


The  numerical  study  below  is  restricted  to  values  of  a.  b,  c,  v 


as  obtained  for  torus  networks  (Eq.  (3.50)),  although  such  a study 


could  easily  be  repeated  for  other  networks  which  belong  to  the  family 


nibi 


i 

t 


r ] 


3. 3. 3.1  Evaluation  with  an  WC  Resulting  in  a Minimal  Table  Length 
The  expression  of  E has  already  been  derived,  in  Eq.  (3.41), 
for  an  WC  which  results  in  a minimal  table  length  A,  Eq.  (2.6).  If 
we  consider  torus  networks,  then  from  Eq.  (3.50)  E becomes 


2 

2N 

(N  - 1)N1/2 

„ N3'2  - N 

3/ 2m 

9 

N3/2m 


N 


n1/2  _ Nl/2m 
N1/2m  - i 


+ 2 


N - N1/m 

^TT 


(m  - 1) 


fv 


(3.52) 


The  above  equations  provide  a parametric  representation  of  E as  a func- 
tion of  A/N.  m acts  as  the  coupling  variable  in  that  representation. 

By  letting  m vary  from  1 to  InN  we  obtain  all  the  possible  values  of 
A/N;  and  subsequently  for  each  value  of  A/N  we  obtain  the  corresponding 
value  of  E.  The  above  range  of  m is  chosen  in  accordance  with  the 
results  obtained  in  Chapter  2 (refer  to  Proposition  2.2  and  Fig.  2.5); 
and  also  in  accordance  with  the  fact  that  E is  an  increasing  fiaiction 
of  m (this  fact  is  obvious  from  the  proof  of  Proposition  3.2). 

Numerical  results  are  presented  in  a set  of  figures  as  follows: 
Fig.  3.5  illustrates  the  behavior  of  E with  respect  to  A/N  and  for 
several  values  of  N.  We  observe  that  an  original  substantial  table 
reduction  can  be  achieved  for  small  values  of  E,  i.e.,  for  a small  drop 
in  performance.  However  if  we  try  to  reduce  A/N  to  values  close  to  its 
global  minimum,  Eq.  (2.19),  then  E increases  shaiply.  Fig.  3.5  also 
illustrates  the  limiting  behavior  of  the  IHR  schemes  (see  Proposition 
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3.4)  whereby  as  N becomes  larger,  more  gains  can  be  obtained  for  a 
lesser  loss  in  performance.  This  property  is  shown  by  the  fact  that 
the  curves  E versus  A/N  stay  flat  on  the  A/ N axis  for  larger  intervals. 
The  linear  scale.  Fig.  3.6,  provides  a better  illustration  of  the  above 
phenomenon. 

Figs.  3.7  - 3.8  show  the  behavior  of  1/(1  + E)  with  respect  to 
A/N.  The  function  1/(1  ♦ E)  represents  a lower  bound  over  h/hc  (see 
Eq.  (3.44)).  These  figures  exhibit  properties  similar  to  the  previous 
ones. 

Finally,  Fig.  3.9  shows  how  much  table  reduction  can  be  ob- 
tained for  a given  "tolerance"  E as  a function  of  the  size  N.  The 
concentration  of  the  curves  for  1 < E 5 (Note  from  Eq.  (3.51)  that 
E ■ 5 is  the  maximw  error  for  torus  networks.)  again  shows  that  beyond 
• certain  point  the  gains  in  table  length  can  only  be  achieved  at  the 
expense  of  large  losses  (large  E).  However  in  the  range  0 to  1 for  E 
considerable  gains  can  yet  be  obtained.  For  that  range  of  E as  shown  in 
Fig,  3.10,  the  corresponding  range  of  the  number  of  levels  m is  limited 
to  fairly  small  values.  The  curves  in  Fig.  3.10  represent  the  values  of 
a which  are  co^>uted  from  Eq.  (3.52)  for  a given  E and  N,  and  which 
served  to  find  the  value  of  A/ N for  a given  tolerance  E (see  Fig.  3.9). 
The  *«ct  that  n is  considered  to  be  a real  variable  will  be  interpreted 
in  the  next  section. 

Moreover,  in  Section  2. 3. 2. 4,  we  noticed  that  most  of  the 
table  reduction  is  obtained  for  small  values  of  a.  We  conclude  that 
the  MIR  schemes  operating  with  a small  number  of  levels  2 <:  m £ 4 
yield  substantial  table  reduction  for  a relatively  small  increase  in 
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: 


path  length. 


which  is  a straightforward  consequence  of  Proposition  2.1.  In  what 
follows  we  as suae  that  l always  satisfies  Eq.  (3.54). 

Problem  3.53  can  be  easily  solved  nuoerically  for  a * 2 and 
a * 3;  beyond  that  it  becomes  quite  onplicated  and  no  effort  has  been 
expended  in  that  direction.  However,  as  noticed  earlier,  m « 2,  3 will 
capture  most  of  the  realistic  table  reductions  for  a fairly  large  range 
of  N (N  up  to  10  , 10  ) . Two  cases  to  consider  are 


m ■ 2 

The  objective  function,  Eq.  (3.40)  and  Eq.  (3.50),  is 


E2  * 4 1/2  (N  " V(nl/2  ' V (3.55) 

L (N  - 1)NA/^  1 1 

Since  the  bound  E is  a decreasing  function  of  SL  (see  Fig.  3.5)  then  at 
optimality  the  constraint  Hj  + n2  < £ is  tight.  Hence  nj  n2  are  such 
that 


( ni  + n2  - 1 
( Dln2  * N 


whose  solution  is  the  pair 


(3.56) 


Finally,  the  optimal  solution  can  be  numerically  obtained  by  choosing 
the  pair  n1#  n2  which  minimizes  E2  and  satisfies  Eq.  (3.56). 

Fig.  3.11  shows  the  plots  of  the  minimum  E2  (denoted  by  E2  on 
the  graph)  with  respect  to  Jl/N  and  for  a set  of  values  of  N.  Also 
represented,  is  the  boimd  E,  Eq.  (3.52),  obtained  with  a vector  n which 
satisfies  Proposition  2.1  (i.e.,  - N1/m  k - 1,  ...,  m).  The  main 
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Figure  3.11. 


Mitin  Bounds  on  th« 
Oiler  at*  m,  1 <m<2. 


in  Path  Length  with  a Continuous  and  a 


observation  is  that  E is,  in  general,  a slightly  larger  bound  than  E2. 

For  large  values  of  fc/N  or  N,  E2  and  E are  very  small  and  close  to  each 

-1/2 

other.  They  are  equal  at  1/ N ■ 2N  ' which  corresponds  to  the  mini- 
mum table  length  at  m ■ 2 (nj  ■ n2  * v/n)  . 


m * 3 


Similar  to  the  above,  at  optimality  the  first  constraint  is 


tight,  hence 


nl  + n2  + n3  * 1 


nln2n3  “ N 


(3.57) 


Then  for  any  feasible  vector  n and  for  a given  n3  > 0,  the  pair  n^  n2 
(or  n2,  nj)  must  be  equal  to 


-"3>2 


(3.58) 


Also  the  expression  of  the  bound  for  a feasible  N is  (Eqs.  (3.40), 


(3.50)  and  (3.57)), 


(N  - 1)NJ 


(N  - n )(nj/2-  1)  ♦ 


(•  ■ s-J(<¥ 


(3.59) 


Note  that  in  Eq.  (3.59),  n2  was  replaced  by  N/n^j.  Then  for  a given 
n3,  the  minimus  of  Ej  can  be  found  by  evaluating  E3  for  the  two  possible 
values  of  n^,  Eq.  (3.58).  Afterward  we  minimize  the  result  over  n3> 

The  results  of  this  entirely  numerical  procedure  are  shown  in  Fig.  3.12, 
(where  the  minimum  is  denoted  by  Ej)  Fig.  3.12  also  shows  the  bound  E 
Eq.  (3.52)  for  a continuous  m (1  m 3).  The  curves  for  the  minimum 
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E2  are  also  shown  here. 


«,i/3 

Notice  that  E • E^  for  4/N  ■ — ^ — which  corresponds  to  the 
minimum  table  length  for  a 3-level  hierarchy.  Similar  observations  as 
above  apply  here.  In  general  for  a given  i/N,  E is  slightly  worse  than 
E2  and  E3»  excePt  for  large  values  of  4/N  (i.e.,  small  table  reduction) 
where  all  the  curves  are  quite  close  to  each  other,  with  sometimes  E 
even  a bit  better.  Also,  the  lower  envelope  of  E2  and  E^  corresponds  to 
the  overall  minimum  of  Problem  3.53  where  m is  restricted  to  either 
2 or  3. 

The  above  conparison  of  E to  E2  and  Ej  leads  us  to  the  follow- 
ing important  remark  as  to  the  utilization  of  a continuous  number  of 
levels  m. 


Remark : Continuous  m 

The  fact  that,  in  general,  E is  slightly  worse  than  E_  or  E. 
will  allow  us  to  always  use  an  MIC  which  results  in  a minimal  table 
length  (4  * nN  ^m)  and  whose  number  of  levels  is  continuous,  and  still 
evaluate  a worst  case  performance  of  the  MIR  schemes.  In  other  words, 
in  the  sequel  the  MIR  schemes  will  always  be  characterized  by  E and  l 
as  given  in  Eq.  (3.52). 

A non-integer  value  of  m may  be  interpreted  as  explained  in 
the  following  example.  Assume  that  we  need  to  operate  a network  of 
size  N with  an  MIR  but  with  an  allowed  maximum  increase  in  path  length 


equal  to  Eq.  For  those  values  of  Eq  and  N we  can  find  the  minimum 
table  length  4q  (as  in  Fig.  3.9)  and  the  corresponding  number  of  levels 
1/m0 

V ^0  = moN  • Now  assume  that  ji>q  *>  1.5;  then  we  can  choose  a 
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nuaber  of  levels  a * 2 and  a vector  n which  minimizes  E ^ (Problem  3.5 
with  m * 2 and  t * Aq) . The  ainiaua  value  of  E ^ is,  in  general, 
slightly  better  than  EQ;  hence  the  new  >HC  with  a discrete  a satisfies 
the  maximum  tolerance  allowed. 

The  above  remark  is  only  true  for  1 _<  a < 3 but  as  noticed 
previously,  that  range  of  a yields  aost  of  the  table  reduction  for 
practical  puxposes.  Also,  froa  the  shape  of  the  curves  in  Fig.  3.12, 
it  appears  that  these  results  could  be  extended  to  higher  values  of  a. 

3.4  Conclusion 

In  this  chapter  we  exaained  the  effect  of  hierarchical  routing 
on  network  path  length.  Bounds  were  derived  to  evaluate  a aaxiaua 
increase  in  path  length  for  a given  table  reduction.  Furtheraore,  the 
bounds  deaonstrate  that  no  significant  increase  in  path  length  need  be 
incurred  in  the  liait  of  a very  large  network. 

The  reduction  in  table  length  means  that  aor<*  channel  capacity 
and  storage  are  available  for  the  transaission  of  data  traffic  in  the 
network.  However,  those  gains  were  obtained  at  the  expense  of  longer 
paths  in  the  network.  It  is  then  natural  to  evaluate  the  performance 
of  the  hierarchical  routing  in  terms  of  delay  and  throughput,  and  to 
define  the  region  of  N where  clustering  becomes  economical . These 
questions  are  the  object  of  the  next  chapter. 
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CHAPTER  4 


STOCHASTIC  PERFORMANCE  EVALUATION 
OF  THE  HIERARCHICAL  ROUTING 

The  previous  chapters  (2,  3)  demonstrated  that  enormous  table 
reduction  can  be  achieved  through  hierarchical  routing.  A shortcoming 
of  those  gains  was  found  in  the  increase  of  the  network  path  length. 
This  means  that  the  nodal  storage  and  channel  capacity  recovered  as  a 
result  of  the  table  reduction  may  have  to  be  partially  or  completely 
(or  more)  paid  back  to  handle  the  excess  traffic  caused  by  longer  net- 
work paths.  In  this  chapter  we  are  interested  in  the  trading  relations 
among  the  table  reduction,  the  nodal  storage,  the  channel  capacity,  the 
network  sixe,  the  throughput  and  the  delay.  Several  queueing  models 
are  developed  to  capture  and  exhibit  the  interrelationships  among  these 
variables.  The  models  demonstrate  that  for  some  reasonable  cost  and 
performance  constraints  and  for  a class  of  symmetrical  and  distributed 
networks,  the  non -hierarchical  routing  becomes  infeasible  for  N (net- 
work sixe)  beyond  some  "critical”  value;  whereas,  on  the  other  hand, 
they  show  that  hierarchical  routing  operating  with  an  appropriate  table 
length  is  capable  of  maintaining  a fairly  good  network  performance  for 
fairly  large  values  of  N. 

In  what  follows,  first  we  present  the  analysis  of  network  per- 
formance under  fixed  routing,  and  a fundamental  assumption  upon  which 
we  will  build  the  models  mentioned  above.  This  assuaption  typically 
results  in  a mapping  of  hierarchical  and  non -hierarchical  adaptive 
routing  into  deterministic  (fixed)  routing.  Then  four  models  will  be 
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gradually  daveloped  to  deal  with  different  s«d>sets  of  the  variables 
above;  listed  by  order  of  complexity,  they  are: 

Model  with  no  updates  and  no  storage  1 imitation 
Model  with  updates  and  no  storage  limitation 
Model  with  no  updates  and  with  storage  1 ini tat ion 
Model  with  ipdates  and  with  storage  limitation 

4. 1 General  Considerations 

The  objective  here  is  to  first  introduce  the  original  nodel 
developed  by  Klein rock  [KLEI  64]  for  delay  analysis  in  stOTe-and- forward 
(S/F)  coeputer  networks.  Then  we  introduce  a class  of  symmetrical  net- 
works and  a main  assumption  which  will  serve  as  a framework  for  the 
four  models  to  be  developed  in  this  chapter. 

4.1.1  Oolay  Analysis  in  S/F  Computer  Networks:  Kleinrock's  Model 

As  described  in  Section  2.1,  in  a S/F  net,  a message  enters 

the  net  at  the  source  node,  visits  several  nodes  along  the  comsuni cation 
path,  and  leaves  the  net  at  the  destination  node.  The  tine  a message 
spends  at  each  node  is  the  sum  of  nodal  processing  time  (for  routing 
purposes),  of  queueing  time  due  to  interfering  messages,  and  the  trans- 
mission time.  A very  important  performance  measure  of  an  S/F  net  is 
the  total  average  delay,  T,  a message  spends  in  the  network.  T is 
then  defined  as 


where:  ■ average  Message  rate  firm  source  i to  destination  j 

z^  • average  nessage  delay  from  i to  j 
Finally  V is  the  total  input  rate  (throughput). 


r • £ Yu 

i.jeS 


(4.2) 


A straightforward  application  of  Little's  result  leads  to  the 
expression  of  T in  terns  of  the  individual  channel  delays  [KLEI  64], 


NA  K 


(4.3) 


where:  • average  traffic  rate  on  channel  i 

tj  ■ average  nodal  processing  plus  queueing  plus 
transmission  on  the  i**1  channel. 

Uhfbrtimately,  we  are  not  able  in  general  to  evaluate  t^  and 
Ai.  However,  if  we  make  the  following  assumptions  [KLEI  64]: 

a.  external  Poisson  arrivals 

b.  exponential  nessage  length  distribution  (identical  for 
all  messages) 

c.  single  packet  messages 

d.  error  free  channels 

e.  no  nodal  delay 

f.  independence  assunption 

g.  deterministic  routing 

h.  infinite  nodal  storage 

With  such  assumptions  (to  be  discussed  later),  the  S/F  net 
can  be  modelled  as  a network  of  queues  of  the  Jackson  type  [JAOC  57]. 
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Discussion  of  the  Above  Assumptions 


Discussions  of  the  above  assumptions  can  be  found  in  [KLEI  64], 
[FULT  72],  [GERL  73A],  along  with  some  further  extensions  of  the  above 
model.  In  particular,  we  will  discuss  the  last  three  assumptions  in 
order  to  motivate  some  of  the  models  to  be  considered  in  this  chapter. 

Independence  Assumption:  [KLEI  64],  [FULT  72] 

Ihis  assumption  is  quite  acceptable  as  long  as  the  network 
does  not  contain  long  chains  with  no  interfering  traffic. 

Deterministic  Routing  Assumption: 

As  mentioned  in  Section  2.1.1,  typically,  a network  is  designed 
using  deterministic  routing  [GERL  73A]  and  then  operated  using  adaptive 
routing.  Consequently  the  validity  of  this  assumption  depends  criti- 
cally on  the  difference  of  behavior  between  deterministic  and  adaptive 
techniques.  A thorough  investigation  on  this  subject  can  be  found  in 
[FULT  72].  Fultz  finds  a close  agreement  between  the  two  techniques. 

He  showed  in  a 19-node  net  application  that  the  difference  in  delays 
obtained  with  a well  chosen  adaptive  technique  and  with  a near- optimal 
deterministic  technique  is  less  than  5 to  10%  almost  until  saturation. 
The  "adaptive"  delay  tends  to  be  higher,  mainly  because  of  the  line 
overhead  utilized  by  the  update  traffic. 

Fultz's  study  is,  of  course,  dependent  on  the  particular 
adaptive  policy  and  network  considered.  However,  it  demonstrates  that 
adequate  adaptive  policies  can  be  devised  to  achieve  a performance  very 
close  to  that  of  a near-optimal  deterministic  policy. 

Further  refinement  of  the  above  assumption  can  be  realized  by 


including  the  line  ovexhead  due  to  the  update  traffic.  This  consider- 
ation will  become  crucial  when  dealing  with  large  nets,  as  we  will  see 
later. 

Infinite  Storage  Assumption 

Nodal  storage  limitation  can  cause  blocking  and,  consequently, 
retransmission  (ARPANET)  or  loss  of  messages,  hence  an  increase  in  delay 
or  a decrease  in  throughput.  Moreover,  it  can  lead  to  serious  network 
degradation  and  even  deadlocks  [FULT  72],  [KAHN  71],  [KLEI  74]. 

Zeigler  [ZEIG  71]  investigated  some  theoretical  aspects  related 
to  the  dynamics  of  cliques  of  blocked  nodes  in  a symmetrical  network 
environment. 

It  is  generally  accepted  [FRAN  70],  [COLE  71],  that  with  rea- 
sonable storage  size,  the  above  assumption  is  fairly  accurate.  However 
this  assumption  becomes  critical  if  the  nodal  storage  becomes  small. 

This  situation  is  very  likely  to  occur  in  a large  network  environment 
if  the  routing  tables  are  not  reduced  to  a reasonable  length.  A model 
is  presented  in  Section  4.4  to  precisely  deal  with  this  question. 

4.1.2  A Class  of  Symmetrical  Networks 

The  delay  analysis  performed  in  the  above  model  relies  on  the 
knowledge  of  the  input  rates  (A^'s)  to  the  individual  channels  in  the 
network.  As  mentioned  earlier,  the  A^'s  can  be  determined  once  we  know 
the  deterministic  routing  policy  and  the  traffic  requirement.  Moreover, 
if  we  know  the  channel  capacities  then  the  A^'s  can  be  computed  to  lead 
to  a minimal  delay  T [FRAT  73]. 
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A shortcoming  of  the  above  numerical  procedures  is  that,  in 
general,  they  hide  the  interrelationships  existing  among  the  different 
design  variables  (traffic  requirement,  channel  capacities,  network 
topology  and  average  delay).  Fortunately,  for  some  symmetrical  networks 
(see  below)  a simple  analytical  relationship  exists  among  the  above 
variables. 

The  class  of  nets  to  be  considered  in  this  chapter  is  composed 
of  all  the  nets  which  belong  to  the  family  of  nets  presented  in  Section 
3.3.1,  and  which  also  satisfy  the  following  properties. 

i.  All  nodes  are  equivalent  with  respect  to  the  topology 
of  the  network.  Hence  they  are  of  equal  degree,  R. 

ii.  All  channels  are  of  equal  capacity  C. 

iii.  All  external  input  traffic  rates  are  equal,  i.e., 

Yjk  - Y * j.  k e S (4.9) 

As  an  example,  torus  nets  (see  Appendix  A)  fall  into  this  category. 

The  above  properties  lead  to  a simple  delay  expression. 

4.1.2. 1 Average  Delay 

For  this  class  of  nets,  the  following  relations  exist: 

Number  of  (simplex)  channels:  NA  = R • N (4.10) 

Total  external  traffic:  T ■ N(N  - l)y  (4.11) 

Furthermore,  it  is  obvious  that  with  this  particular  topologi- 
cal structure,  capacity  assignment  and  traffic  requirement,  the  optimal 
flow  assignment  [KLEI  64],  [GERL  73A]  is  a shortest  path  routing.  The 
selection  of  the  particular  shortest  paths  (in  case  that  more  than  one 
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II 


1 


exists)  must  result  in  perfectly  balanced  flows,  i.e.. 


X.^  = X i = 1,  . . . , NA 


(4.12) 


Consequently  the  network  path  length  n becomes  the  average  shortest 
path  length  h,  defined  in  Appendix  A,  Eq.  (A.2),  then  from  Eq.  (4.6) 


A V 

f = h 


(4.13) 


Also  the  total  internal  traffic  A (Eq.  (4.7)),  becomes 


A = NA  X 


(4.14) 


Combining  the  two  equations  above,  we  arrive  at 


X = h 


NA 


(4.15) 


If  we  let  t denote  the  average  delay  on  any  channel,  then  the  total 
average  delay  becomes,  Eq.  (4.3), 


T = ht 


(4.16) 

Moreover,  as  a consequence  of  Eqs.  (4.4),  (4.5),  (4.12)  and  (4.15) 

1 


VC  - X 


and 


T-W-r 

X ' NA 


(4.17) 

(4.18) 


This  is  the  result  we  were  aiming  at,  which  simply  relates  the  delay  T, 
the  traffic  T,  the  channel  capacity  C and  the  network  path  length  h. 

Eq.  (4.18)  shows  that  the  net  is  equivalent  (with  respect  to 
delay)  to  a single  m|m|i  queue  with  an  input  rate  of  T/NA  and  a service 


•i 
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rate  pC/h.  This  observation  leads  to  the  definition  of  the  network 
utilization 

(4-19) 

The  above  definition  will  be  used  in  the  rest  of  the  chapter,  mainly 
for  normalization  purposes. 

4. 1.2.2  A Scaling  Scheme 

Since  the  main  objective  here  is  to  study  routing  in  large 
nets,  it  is  necessary  to  specify  the  structure  of  those  large  nets  with 
respect  to  the  size  N,  in  some  continuous  way.  Any  such  specification 
will  be  referred  to  as  a scaling  scheme  (strategy). 

As  the  network  grows,  a main  objective  of  a scaling  strategy 
could  be  to  maintain  the  same  average  delay  T for  a reasonable  increase 
of  the  total  traffic  T and  of  the  network  cost  (channel  capacity  cost) . 
The  total  traffic  T may  be  reasonably  assumed  to  increase  linearly  with 
the  number  of  nodes.  Which,  due  to  the  uniform  traffic  condition 
(Yjk  * Y),  is  equivalent  to  assuming  that  the  total  input  rate  per  node 
is  maintained  constant.  Let  g denote  the  total  input  rate  per  node 
divided  by  the  degree  of  a node, 

g & (4.20) 

Note  that  y will  be  adjusted  in  order  to  maintain  g constant.  Conse- 
quently and  because  of  Eqs.  (4.10),  (4.11)  and  (4.18), 

r - NRg  - NAg  (4.21) 
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T = (yC/h  - g) 


(4.22) 


In  order  to  maintain  a constant  delay  TQ,  with  the  above  traf- 
fic requirement,  the  channel  capacity  must  be  such  that:  (Eq.  (4.18)). 


h / 1 \ 

"(V  g) 


(4.23) 


which  says  that  C must  grow  like  the  network  path  length  h.  If  we  let 
Cq  be  a constant  capacity,  then  the  appropriate  capacity  scaling  is 


C * hC, 


(4.24) 


Notice  that  the  ratio  of  total  capacity  to  the  total  traffic  is  equal 
to  hC_/g.  This  indicates  that  for  a linear  growth  of  N and  T,  the 

JZ 

capacity  required  per  message  need  only  grow  like  h (e.g. , h * y-  for 
torus  nets) . 

From  the  above  considerations  emerge  two  scaling  schemes: 


Primal  scaling: 


C « hC„ 


T * T„ 


(4.25) 


Dual  scaling: 


hC0 
(r  = NAg 


(4.26) 


The  outcome  of  the  primal  scaling  is  a traffic  T = NAg  where 
g * yCQ  - 1/Tq,  and  conversely  the  outcome  of  the  dual  scaling  is  a 
constant  delay  T * (yC^  - 8q)_1.  Such  outcomes  will  not  be  true  when 
dealing  with  network  models  which  take  into  account  the  updates  and/or 
the  storage  limitation. 
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. 3 Continuous  Modelling  of  the  Hierarchical  Routin; 

The  objective  here  is  to  summarize  the  key  variables  governing 
the  behavior  of  hierarchical  routing  in  a continuous  (homogeneous) 
approach. 

Recall  that  a one  level  hierarchical  routing  (CER  or  OBR)  is 
simply  the  non-clustered  routing  (NCR)  defined  in  Chapter  3.  As  a con- 
sequence and  throughout  this  chapter  we  will  refer  to  the  three  routing 
schemes  (CER,  OBR,  NCR)  as  hierarchical  routing  schemes  with  the  under- 
standing that  m = 1 refers  to  the  NCR.  Moreover,  as  observed  from 
previous  considerations,  the  underlying  clustering  structure  is  com- 
pletely defined,  knowing  the  number  of  levels  m.  More  specifically, 
the  value  of  m determines  exactly  the  length  of  the  table  l * mN1^01; 
subsequently,  we  can  use  the  remark  at  the  end  of  Section  3. 3. 3. 2 to 
discretize  m and  to  find  the  corresponding  degree  vector  n.  Notice 
that  if  m » 1 then  i - N,  i.e.,  a full  length  table  is  utilized,  which 
corresponds  to  an  NCR.  To  summarize,  the  underlying  clustering  struc- 
ture of  the  hierarchical  routing  (OBR,  CER,  NCR)  is  characterized  by 
the  single  continuous  variable  m.  For  that  reason  m will  be  referred 
to  as  the  degree  of  clustering 

Chapter  3 showed  that  the  reduction  in  table  length  from  l = N 
to  l « mN^m,  due  to  the  hierarchical  routing,  is  accompanied  by  an 
increase  in  path  length  from  h to  h£.  h£  is  characterized  by  Prop- 
osition 3.1,  which  subsequently  provides  us  with  the  basis  for  an 
algorithmic  confutation  of  h , given  a specific  network  and  a specific 

V 

MIR  scheme.  Fortunately,  the  bounds  derived  in  Chapter  3 will  allow 
us  to  undertake  a worst  and/or  best  case  analytical  performance 
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evaluation  of  the  hierarchical  routing  for  the  class  of  networks  con- 
sidered here.  Recall  that  those  networks  constitute  a subset  of  the 
family  studied  in  Chapter  3,  hence  we  will  be  able  to  use  the  explicit 
expressions  for  the  relative  bounds  E or  F as  derived  in  Eqs.  (3.41) 
and  (3.47). 

Notice  that,  similar  to  the  above,  the  bounds  rely  on  the 
continuous  behavior  of  m,  and  more  importantly,  they  are  tight  at  m = 1 
These  last  observations  will  allow  us  to  use  the  above  continuous 
approach  in  dealing  with  the  hierarchical  and  non-hierarchical  routing 
schemes.  The  comparison  between  those  two  schemes  sinply  reduces  to 
the  comparison  of  a hierarchical  routing  with  m * 1 (NCR)  to  one  with 
m > 1 (CER  or  OBR) . 

In  summary,  knowing  the  degree  of  clustering  m,  the  increase 
in  path  length  is  characterized  by  the  relative  bounds  E or  F.  In 
general  E will  be  chosen,  unless  specified  otherwise.  Hence, 

h < hc  < (1  + E)h  (4.27) 

and  m * 1 “>  E = 0 

Even  with  the  above  simple  specifications  of  the  hierarchical 
adaptive  routing  scheme  (i.e.,  m,  E)  the  queueing  analysis  is  still 
far  too  complicated.  This  is  true  for  any  adaptive  scheme  because  of 
its  dynamic  nature.  In  the  face  of  these  difficulties,  we  will  rely  on 
observations  made  about  the  deterministic  routing  assumption  in  Section 
4.1.1  to  justify  the  modelling  of  a hierarchical  adaptive  routing  by  an 
"equivalent"  deterministic  routing;  hence  the  assumption  below: 


iii  i jin  wj.m 

Assumption  4.1 

a.  The  performance  of  an  adaptive  hierarchical  routing  is 
the  same  as  that  of  a fixed  routing  policy  whose  routes  satisfy 
Proposition  3.1.  That  is,  the  length  of  the  "fixed  routing"  paths  are 
equal  to  the  minimum  estimated  path  lengths  as  obtained  with  an  MIR. 

b.  The  fixed  routing  specified  above  and  operating  on  the 
class  of  symmetrical  nets  considered  here,  results  in  equal  loads  on 
all  channels. 

Part  (a) , again  motivated  by  an  earlier  remark  on  deterministic 
routing,  will  become  more  accurate  when  we  include  in  the  model,  based 
on  fixed  routing,  the  line  and  storage  utilization  due  to  the  adaptive 
routing.  Moreover,  if  the  main  objective  is  to  compare  hierarchical 
with  non-hierarchica 1 routing,  then  this  assumption  appears  to  be  quite 
acceptable. 

Part  (b)  is  motivated  by  the  quite  symmetrical  structure  of 
the  networks  considered  here,  and  also  by  the  fact  that  the  main  objec- 
tive of  an  adaptive  policy  is  to  balance  the  flows  over  all  the  channels 
in  the  net. 

Notice  that,  because  of  the  above  assumption,  the  NCR  (MIR 
with  m ■ 1)  is  modelled  by  the  shortest  path  fixed  routing  which,  as 
observed  in  Section  4. 1.2.1,  leads  to  the  optimal  flow  assignment  for 
this  class  of  nets. 

Conclusion: 

As  a result  of  the  above  considerations,  a hierarchical  routing 
is  characterized  by  the  degree  of  clustering,  m,  and  the  path  length. 


hc>  and  can  be  modelled  by  a balanced  fixed  routing  resulting  in  paths 
of  the  same  lengths. 


A Queueing  Model  with  No  Updates  and  No  Storage  Limitation 


In  this  section,  we  still  temporarily  relate  the  gains  obtained 
by  hierarchical  routing  strictly  to  the  relative  table  length  Jl/N.  In 
other  words,  we  consider  an  ideal  situation  where  infinite  nodal  storage 
is  available  and  where  the  line  capacity  used  up  by  the  update  infor- 
mation is  still  negligible.  This  is  similar  to  the  study  in  Section  3.3 
except  that  we  now  intend  to  look  at  the  degradation  of  the  network 
performance  in  terms  of  delay  or  throughput. 

Based  on  the  previous  general  conditions  (Section  4.1)  and 
the  above  remarks,  this  situation  can  be  modelled  by  Kleinrock's  model 
(Section  4.1.1)  where  the  fixed  routing  is  as  specified  in  Assumption 


The  delay  analysis,  for  our  class  of  symmetrical  nets,  is 
reduced  to  the  one  performed  in  Section  4. 1.2.1  except  that  h is  to  be 
replaced  by  hc«  As  a result  the  throughput-delay  relation  becomes 
(Eq.  (4.18)). 


(4.28) 


Notice  the  change  in  notation  from  T to  Tc  and  T to  T^.  The  subscript 
c is  added  mainly  to  differentiate  between  clustered  and  non-clustered 
routing  for  later  comparisons.  The  notation  Tt  T will  be  strictly 
reserved  for  the  NCR  and  Tc,  T£  for  all  hierarchical  routing;  of  course. 
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if  a ■ 1 then  r * T and  T * T.  Similar  to  Eq.  (4.16)  and  Eq.  (4.21) 

C v 

we  will  also  let 

Tc  ■ Vc  • rc  ■ NA«c  <4'29> 


where  t and  g are  now  variables. 

c c 

The  behavior  of  the  hierarchical  routing  may  now  be  studied 
assuming  either  the  primal  or  dual  scaling  schemes  (see  Section  4. 1.2.2) 
in  order  to  specify  the  growth  of  our  class  of  nets. 

The  numerical  applications  throughout  this  chapter  will  assume 
the  coefficients  a,  b,  c,  v as  obtained  fbr  torus  nets  (Eq.  3.50). 


4.2.1  Degradation  in  Throughput  at  Constant  Delay  (Primal  Scaling) 
Recall  that  the  primal  scaling  (PS)  is  characterized  by 
C ■ hCQ  and  T ■ T£  ■ Tq  (Eq.  (4.2S))  and  also  fbr  an  NCR  hc  ■ h,  hence 
from  Eq.  (4.28)  the  ratio  of  throughputs  with  and  without  clustering  is 

rc  hUc0/hc  - i/t0 
T"  uc0-l/r0 

from  the  above  expression  the  effect  of  clustering  can  be  seen  in  the 
reduction  of  the  line  capacity  by  a fraction  h/hc. 

Me  can  now  directly  apply  Proposition  3.4  to  state  a similar 
asy^>totic  result.  Namely,  as  N,  the  number  of  nodes,  goes  to  infinity 
the  throughput  at  constant  delay,  obtained  with  an  MIR  (CER,  OBR)  with 
a fixed  m,  approaches  that  of  an  NCR  while  the  relative  table  length 
(A/N)  approaches  zero. 

As  for  the  continuous  behavior  of  T /T  versus  A/N  we  use 

c 

Eq.  (4.27)  to  derive  the  bounds  below. 
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LB(  ) is  used  to  denote  the  lower  bound  on  the  variable  in  parenthesis; 
similarly,  UB(  ) will  be  used  to  denote  an  upper  bound. 

Let  us  examine  the  behavior  of  I^/r  by  plotting  its  bounds 
given  in  Eq.  (4.30).  Recall  that  throughout  this  chapter  Eq.  (3. 52), 
giving  E and  1/ N,  will  be  used  in  the  numerical  examples.  This  results 
in  the  curves  shown  in  Figs.  4.1  - 4.3.  The  interpretation  of  those 
polts  is  quite  similar  to  that  provided  for  Figs.  3.5  - 3.9  in  Section 
3.3.3.  We  reemphasize  the  fact  that  initial  (substantial)  table  gains 
(small1  m)  can  be  obtained  with  a relatively  small  degradation  in 
throughput.  Whereas  larger  table  reduction  (large1  m)  may  drive  the 
lower  bound  to  zero.  The  asymptotic  property  is  nicely  illustrated  in 
the  sharp  behavior  of  the  lower  bound,  for  a large  N,  in  Fig.  4.2. 

Also,  Fig.  4.3  shows  that  as  N increases  the  cost  incurred  with  a fixed 
relative  table  length  A/N  goes  to  zero. 

We  note  also  that  the  curves  in  Figs  4.1  and  4.2  meet  at  the 
point  A/N  » 1,  LB(r  /T)  » 1.  That  point  corresponds  to  m * 1 where 
the  bound  is  tight. 


Recall  from  Chapter  2 that  most  of  the  table  reduction  is  obtained  for 
small  values  of  m,  and  the  remaining  reduction  up  to  the  global  minimum 
table  length  is  obtained  with  quite  a bit  larger  m (up  to  m#). 


,/n 


LBII^/D;  Modal  with  no  Updates  and  no 
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4.2.2  Degradation  of  Delay  at  Constant  Throughput  per  Node  (Dual 


Scaling) 

We  now  intend  to  study  the  dual  behavior  by  applying  the  dual 

scaling,  whose  purpose  is  to  maintain  r/NA  constant  and  to  let  T vary 

c 

Let  g * g * g«  * r/NA  and  C = hC_;  then  the  ratio  of  delays  T /T  is 


Immediately  we  obtain  the  dual  asymptotic  result  whereby  as  N goes  to 


infinity  the  network  delay  obtained  with  an  MIR  (m  fixed)  approaches 
that  of  an  NCR,  while  the  relative  table  length  goes  to  zero. 

As  for  the  general  behavior  of  T /T  versus  J/N,  from  Eq.  (4.27) 


The  inverse  of  the  above  bounds  are  plotted  in  Fig.  4.4,  in  order  to 
show  the  behavior  of  T/Tc;  dual  properties  are  exhibited.  We  also  note 
that  in  some  regions  pCg/(l+E)  < g^,  hence  the  corresponding  degree  of 
clustering  m is  infeasible. 


A Queueing  Model  with  Updates  and  No  Storage  Limitation 


Our  previous  models  expressed  the  gains  obtained  from  the 
hierarchical  routing  strictly  in  terms  of  the  relative  table  length 
£/N.  They  left  out  the  further  (significant)  savings  incurred  in  line 
capacity  and  nodal  storage.  As  a result  of  this  quite  idealistic  sit- 
uation of  abundance  in  capacity  and  storage,  the  NCR  showed  a better 


T/Tc 


Figure  4.4.  Degradation  in  Delay  at  Constant  Throughput  par  Node; 
Model  .vith  no  Updates  and  no  Storage  Limitation. 
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network  performance  than  the  MR  except  at  the  limit  of  very  large  nets 
where  they  were  quite  equivalent  even  for  substantial  table  reductions. 
This  last  (asymptotic)  property  makes  us  believe  intuitively  that  in  a 
less  idealistic  situation  (storage  and  capacity  limitations)  the  MR 
with  jthe  rigjit  degree  of  clustering  will  eventually  (in  terms  of  size) 
outperform  the  NCR.  Moreover  we  can  already  observe  that  if  no  cluster- 
ing is  introduced,  the  linear  growth  of  a full  length  table  (N  entries) 
with  N will,  at  a certain  point,  use  up  all  the  nodal  storage  and  the 
line  capacity  (for  the  exchange  of  updates) , hence  driving  the  network 
throughput  to  zero. 

In  this  section  we  intend  to  account  for  the  traffic  generated 
by  the  routing  updates  (in  the  evaluation  of  the  X^'s)  while  keeping 
the  infinite  storage  assumption. 

As  noticed  earlier,  the  average  delay  in  our  class  of  symmet- 
rical nets  is  very  simply  related  to  the  average  delay  at  any  channel 
(T  * ht) ; therefore,  we  will  first  analyze  a single  channel  and  then 
generalize  to  the  whole  net. 

4.3.1  Priority  Model  for  a Channel 

A simple  and  realistic  Head  £f  Line  (HOL)  model  [KLEI  76]  is 
considered  here,  mainly  to  capture  the  effects  of  updates  on  the  average 
time  spent  by  a data1  message  waiting  to  be  transmitted  on  a channel. 

We  assume  that  updates  are  originated  at  regular  intervals  of  time 
(motivated  by  ARPANET) . Aperiodic  updates  may  be  model led  by  a "no 

1 A data  message  is  differentiated  from  an  update  message. 
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update"  model  (Sections  4.2,  4.4)  or  by  a certain  distribution  governing 
their  generation  tines.  The  latter  possibility  can  be  easily  included 
in  the  nodel  below  for  such  distribution  as  the  Poisson  distribution. 
(However,  no  woTk  has  been  done  in  that  direction.) 

In  summary,  our  model  for  a channel  consists  of  a single  queue 
operated  with  a HOL  priority  discipline  and  the  following  traffic 
charact eri sties. 

i.  Update  traffic:  Deterministic  arrival  process  of 

rate  Xu<  Constant  message  length  l/yu  [kbt/messg]. 

ii.  Data  traffic:  Poisson  arrival  process  of  rate  X. 

Exponential  message  length  of  mean  1/p  [kbt/messg]. 

iii.  Queue  discipline:  HOL  preemptive  resume  between  data 

and  update  traffic,  with  a higher  priority  for  updates. 
FCFS  (first-come-first-serve)  within  each  priority. 

iv.  Qiannel  capacity:  C [kbt/sec]. 

The  "preemptive  resume"  assunption  in  (iii)  is  introduced  to  further 
simplify  the  analysis  of  the  model,  which  otherwise  becomes  quite 
conplex. 

The  above  model  is  slightly  different  from  the  one  analyze  1 by 
Kleinrock  [KLEI  76]  where  he  considers  the  arrival  processes  of  all  the 
types  of  customers  (messages)  to  be  governed  by  a Poisson  distribution. 
However,  his  methodology  can  still  be  used  here  in  order  to  derive  the 
average  time  in  system  for  a data  message. 

With  regard  to  the  update  traffic,  it  siaply  sees  a D|D|l 
system;  hence  as  long  as  Xy  < uuC  (Xu  _>  yuC  means  that  more  than  the 
total  capacity  is  required  by  the  updates,  thus  the  routing  becomes 
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infeasible)  there  is  no  queueing  of  update  messages;  whereas,  an  arriv- 
ing data  message  will  incur  a delay  from  the  message  (data  or  update) 
already  in  service,  from  data  messages  already  in  the  queue  and  from 
updates  arriving  during  its  system  time  (see  Fig.  4.5) 


ARRIVING 

DATA 

MESSAGE 


IX2|Xl| 


Y JDATA  OR  UPDATE 


DATA 


Figure  4.5.  State  of  the  Queue  as  seen  by  an  Arriving  Data  Massage. 


Let  us  now  evaluate  the  different  components  involved  in  the  delay  of 
a data  message. 

1.  Delay  due  to  messages  already  in  queue.  Let  8 denote  the 

expectation  operator,  n be  the  number  of  data  messages  (updates  do  not 

join  the  queue)  that  our  arriving  data  message  finds  in  the  queue,  and 

be  the  service  required  by  the  i^  message;  then  the  average  wait 

incurred  is  <?(I.X.). 

v i 

Clearly,  X^,  X^,  ...»  Xr  are  identically  distributed  (expo- 
nential distribution  of  mean  1/y) ; the  question  arises  as  to  the 
distribution  of  Xj,  the  service  (entire  or  remaining)  time  of  the  data 
message  in  the  front  of  the  queue  (see  Fig.  4.5)  which,  as  such,  could 
have  been  preempted  several  times  in  the  past.  Fortunately,  because 
of  our  exponential  distribution  assumption  for  data  messages,  the 
remaining  service  time  is  also  exponentially  distributed  no  matter  how 


many  times  that  message  has  been  preempted.  Hence  <?  (E^X^)  = n/yC, 


where  n is  the  average  number  in  queue  as  seen  by  our  arriving  message. 
Since  data  arrivals  are  Poisson,  then  the  distribution  of  n is  the  same 
as  the  distribution  of  the  number  of  messages  in  the  queue  at  any  arbi- 
trary time.  Therefore,  if  t is  the  average  time  in  system  (queueing  + 
service)  for  a data  message,  then  because  of  the  previous  remark  and 
from  Little's  results,  n = A(t  - 1/yC) . 

2.  Delay  due  to  the  message  in  service.  Let  Y be  the 
residual  life  (remaining  service  time)  of  the  message  in  service.  <f(Y) 
is  the  contribution  in  delay  that  we  need  to  evaluate.  Conditioning  on 
the  type  of  message  in  service,  it  is  clear  that 

<?(Y)  » ^[residual  life  of  data  message]  • ^ 

A 

+ & [residual  life  of  update  message]  • —4r 

■ u 

Hence*  * yj  ’ IiTc 

3.  Delay  due  to  updates  arriving  while  in  system.  A data 
message  spends  an  average  time  t in  the  system.  During  that  time,  on 
the  average,  Aut  update  messages  arrive  and  get  serviced;  hence  the 
average  wait  time  incurred  is  Aut  l/yuC.  Notice  that  even  though 
update  arrivals  are  deterministic,  their  relative  positions  with  respect 
to  data  messages  are  completely  random  due  to  the  Poisson  distribution 
of  data  arrivals. 

Finally  summing  up  all  the  waits  incurred  by  our  arriving 
message  and  its  own  service  time,  we  arrive  at 
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4. i'.2  Network  friodel 

Similar  to  the  previous  motielj  Assumption  4it  results  in  equal 
loads  gf  data  messages  X ii  2 kcr^xi:  Moreover  because  of 

the  periodic  {$&tS  assumption,  all  will  Peceivg  an  ^ual  rate 


of  updates  Xu. 

^iiK  regard  to  the  delay  analysis  for  our  class  of  synnetrical 
networks,  Eqs.  (4.10)  - (iilfe)  are  still  valid  when  we  replace  h fey  hc< 
tising  a similar  notation  as  iii  Section  %\2  (t;  fc;  il;  fee;  i;  tc,  f,  fc, 
if  t we  arrive  at  the  throughput -del ay  relation  feelow  which  charac- 
terizes the  Performance  of  tke  hierarchical  routing. 


±.  rXt  - -M  J_ 

yc  L ycj  yc 


, x x 
+ “AT  + -^L-2  + TTct 

[yc]z  2(yuC)2  V 


If  we  set  i - B in  ffcl  %£%  [i±  , ii  £ neglect  updates) 

then  we  arrive  at  the  original  expression  for  t,  fed-  (4-17).  The 
difference  between  the  two  equations  illustrates  the  effect  of  the 


Recall  that  the  size  of  an  update  message,  l/vu,  is  fixed  and  propor- 
tional to  the  length  l (1  ■ aN1^")  of  the  routing  table.  Hence,  it  is 
of  the  form  l/uu  * cl  where  e is  the  inverse  of  the  nuaber  of  entries 
which  add  to  1 kbt.  As  an  example,  in  the  ARPANET,  an  entry  requires 
16  bits  of  storage,  hence  e » .016  * 1/62. S.  For  further  normalization 
with  respect  to  the  average  data  message,  1/y,  we  choose  e such  that 


(In  the  numerical  exanples  we  will,  in  general,  choose  1/u  * 1 kbt  and 
e « 1/64.) 

The  behavior  of  the  hierarchical  routing  may  now  be  studied 
for  networks  whose  growth  is  governed  by  our  scaling  schemes. 

4.3.3  Performance  Evaluation  of  the  Hierarchical  Routing 

The  primal  and  dual  scaling  schemes,  Eqs.  (4.25)  - (4.26)  need 
further  specifications  as  to  the  choice  of  the  update  rate  Au  in  tens 
of  the  network  parameters. 

4. 3. 3.1  Scaling  of 

Let  us  recall  that  the  main  purpose  of  the  routing  ipdates  is 
to  provide  the  routing  decision  algorithm  with  a good  estimate  of  the 
state  of  the  network.  Since  at  each  update,  exchange  of  information 
(not  necessarily  synchronized  among  all  nodes)  occurs  only  between 
neighboring  nodes,  then  the  propogation  of  a change  occuring  in  a cer- 
tain region  of  the  net  to  another  region  might  require  a number  of 
updates  equal  to  the  distance  separating  the  two  regions.  Consequently, 
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as  the  network  grows  and  if  we  wish  a change,  conveyed  through  the 
exchange  of  updates,  to  reach  renote  areas  within  reasonable  tine,  then 
it  is  necessary  to  increase  the  update  rate  as  N increases.  We  nay  also 
argue  the  "very"  renote  areas  would  not  be  as  concerned  with  that  change 
as  the  closer  ones,  thus  the  update  rate  probably  need  not  increase 
as  fast  as  N.  A realistic  compromise  would  consist  in  the  use  of  higher 
update  rates  (as  N increases)  but  only  to  propagate  less  and  less  infor- 
mation about  a region  as  we  move  away  from  that  region.  This  remark  is 
again  a key  motivation  behind  the  hierarchical  routing. 

From  the  above  considerations  emerge  three  possible  specifi- 
cations for  X . 


constant 


Choice  (i)  represents  a worst-case  condition  thereby  the 
update  rate  is  insensitive  to  the  increase  in  network  size. 


Choice  (ii)  appears  to  be  more  intuitive  since  the  update 
information  needs  on  the  order  of  h (average  path)  periods  to  percolate 
throughout  the  net. 

Choice  (iii)  is  a conpromise  between  the  two  above;  it  indi- 
cates that  routing  information  need  not  percolate  as  fast  in  the  entire 

1/2 

net,  but  only  within  a certain  area  comprising  roughly  N nodes. 

We  consider  these  three  choices  below. 


4. 3. 3. 2 Throughput  at  Constant  Delay  (Primal  Scaling) 

With  the  primal  scaling,  C ■ hCQ  and  T ■ TQ,  the  network 
throughput  is  (Eqs.  (4.34)  and  (4.35)) 


For  any  routing  to  be  feasible,  the  right  hand  side  of  the  above 
equation  must  be  positive. 


Asymptotic  Behavior 

As  the  number  of  nodes  goes  to  infinity  and  under  the  con- 
straint on  \u  below  (Eq.  (4.37)),  for  a hierarchical  routing  to  be 
feasible,  its  number  of  levels  m must  be  greater  than  or  equal  to  a 
certain  value,  mQ.  Moreover,  the  asynptotic  throughput  (Eq.  (4.38)) 
shows  that  at  the  limit,  with  any  feasible  hierarchical  routing 
(m  fixed) , the  effect  of  the  updates  on  the  channel  utilization  becomes 
negligible. 

The  condition  on  is  such  that  there  exists  which 


satisfies 


limit 

N-*» 


c2^n2/"°; 


U 
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(4.37) 


This  condition  is  motivated  by  the  fact  that  the  predominant  term 
(as  N -*•  •)  in  Eq.  (4.36)  must  go  to  zero  as  N -*■  08  for  any  hierarchical 
routing  to  be  feasible.  (Notice  that  we  replaced,  in  that  term,  i.  by 
«N1/m  and  h by  aNV.) 

The  proof  of  the  above  fact  follows  very  simply: 


mm  ' 
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Let  ■ be  fixed  and  ■ > n^;  then  from  Proposition  3.4 , limit  h/h  « 1, 

eAA  **  ■*''*  C 

and  from  £q.  (4.37),  liait  » u ■ 0; 

N • “c 


£mvCo-% 


(4.38) 


which  terminates  the  proof  of  the  above  fact. 

Ihe  above  limiting  throughput  is  equal  to  that  obtained  in  our 

previous  no del  where  no  updates  are  considered.  It  is  a note  realistic 

result  because  now  only  a feasible  routing  (■  _>  ^)  can  achieve  that 

performance.  This  brings  us  to  a consideration  of  the  range  of  mQ  in 

terms  of  the  specific  choice  of  X . Let  A be  of  the  form  A * Nx 

u u u 

(°  ^ x < 1)  j then  for  the  limit  in  Eq.  (4.37)  to  be  true,  ^ must  be 
such  that 


Thus 


~ ♦ x - v < 0 

■o 


0 < ~~  < V - x 

o 


For  the  above  relation  to  be  possible,  x must  be  less  than  v with  this 
condition 

■o  > 

Applying  this  result  to  our  selected  scaling  schemes  for  A , and  using 


u 


the  fact  that  0 < v < 1,  we  arrive  at 

■ 0 -o  mQ  > 2/v  >_  2 
x ■ v infeasible 


* *0 
A ■ A 
u u 


- *X 


»„  ■ *v/2< 


v/2  ->  mp  > 4/v  _>  4 
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As  m consequence  only  the  first  and  third  schemes  may  yield  a feasible 
hierarchical  routing.  Moreover  a non -hierarchical  routing  is  always 
infeasible  at  the  linit  of  very  large  networks. 


The  above  properties  are  illustrated  in  the  numerical  study 


below. 


General  Behavior 


Similar  to  Section  4.2.1,  a lower  and  upper  bound  on  the 
throughput  can  be  derived  using  Eqs.  (4.27)  and  (4.36). 


a(sf)-rr 

*(k)-  “co 


et\  A eV 

u 1 u , . 


cl\  i 


(4.40) 


Let  us  examine  the  behavior  of  T /NA  with  respect  to  N and  m by  plotting 

c 

its  bounds  normalized  by  the  maximum  throughput  (per  channel  T/NA) , 


Eq.  (4.38);  i.e.,  we  will  plot 


LB(rc/NA) 


UB(rc/NA) 


The  values  selected  fbr  the  different  variables  are: 
pC0  - 6 messg/sec 
Tp  ■ .5  sec 
- .07  UC0 
e - 1/64 

Recall  that  E and  l are  given  in  Eq.  (3.52). 
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Three  sets  of  curves  are  shown  in  Figs.  4.6  - 4.8  and  corre- 
spond respectively  to  the  three  specifications  of  Au,  i.e. , Au  - A®  , 

Au  * N1^4A®/2  , Ay  * N^2A®/ 2.  A few  remarks  emerge  from  the 

observations  of  those  figures. 


Remark  1:  Optimal  degree  of  clustering 


Lower  and  upper  bound  envelopes  are  also  plotted  (by  hand)  on 
each  of  those  figures.  As  a result,  hierarchical  routings  with  the 
appropriate  degree  of  clustering  m,  will  achieve  a throughput  (normal 
ized)  in  the  region  comprised  between  the  lower  and  upper  envelopes. 

lhe  optimal  m corresponding  to  a particular  envelope  can  be 
determined  numerically  as  well  as  the  envelope  itself.  Such  an  oper- 
ation can  as  easily  be  done  by  hand  for  the  graphs  presented  here. 

Given  N,  choosing  m on  the  lower  bound  envelope  (Fig.  4.6)  guarantees 
a minimum  throughput  equal  to  the  corresponding  point  on  the  envelope. 
Fig.  4.9  shows  the  relative  table  length  (obtained  with  those  values 
of  m)  respective  to  Fig.  4.6,  and  also  the  discretized  number  of  levels 
(as  suggested  in  Section  3. 3. 3. 2). 


Remark  2:  Feasibility  and  viability 


The  fact  that  the  lower  bound  and  upper  bound,  for  a specific 
set  of  parameters,  become  closer  for  small  values  of  and  meet  at 
Tc  ■ 0 indicates,  first,  up  to  what  size  N a certain  degree  of  cluster- 
ing is  feasible,  and  second,  that  the  points  r * 0 occur  when  the  line 

c 

capacities  are  totally  utilized  by  the  updates. 

Table  4.1  below,  shows  approximate  values  of  the  points  where 


. 


CO 


»H5i 


' ; — 1 


a non-hierarchical  Touting  (m  3 1)  becomes  infeasible  and  also  shows 
the  points  beyond  which  a 2-level  hierarchical  routing  becomes  certainly 
(due  to  a lower  bound)  more  viable.  Notice  that  the  faster  the  rate 
of  update  exchange  Au  is,  the  smaller  those  critical  values  of  N are. 


Points  at  which  NCR 
becomes  infeasible 
(f  ■ 0 , m = 1) 

Points  beyond  which 
clustering  is  better 

A a A0 
u u 

N * 200 

,0  N1/4 

* u — 2" 

= 1000 

= 150 

,0  N1'2 
* u — 2 — 

* 300 

= 100 

Table  4.1  Critical  Values  of  N 


Remark  3:  Asynptotic  behavior 

The  shape  of  the  lower  bound  envelopes  for  Ay  * A0,  N1//4A^/2, 

show  an  initially  decreasing  and  then  slowly  increasing  behavior  with 
respect  to  N;  the  increase  will  eventually  bring  the  curves  closer  to 
their  asysf>tote  1.  However,  for  A^  » N^^^A^y^2,  the  lower  bound  enve- 
lope is  a decreasing  function  of  N which,  as  predicted  earlier,  will 
eventually  reach  zero.  This  means  that  in  the  neighborhood  of  a certain 
size  N,  the  hierarchical  routing  altogether  becomes  infeasible.  For- 
tunately, that  size  is  well  beyond  108! 

4.3. 3. 3 Delay  at  Constant  Throughput  per  Node  (Dual  Scaling) 

Tc/NA  is  maintained  constant,  i.e.,  rc/NA  « gQ,  hence 
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(4.41) 


Again,  applying  Eq.  (4.27),  we  derive  an  upper  and  a lower  bound  on  T 

C 


UB(Tc) 


LB(Tc) 


(4.42) 


(4.43) 


Dual  properties  to  the  primal  scaling  hold  true  here  and  are  illustrated 
in  the  curves  shown  in  Figs.  4.10  and  4.11. 

Notice  also  that  under  the  condition  of  Eq.  (4.37)  and  for  a 
given  m,  m _>  mQ;  then 


limit  T 

M -W  m C 


which  is  the  dual  of  Eq.  4.38,  and  is  used  to  normalize  the  values  of 
Tc  in  the  plots. 


4.4  A Queueing  Model  with  No  Updates  and  with  Storage  Limitation 

For  large  networks,  the  earlier  infinite  nodal  storage  asstup- 
tion  becomes  less  reasonable.  The  purpose  of  this  section  is  to  develop 
and  analyze  a Kleinrock-like  model  which  takes  into  consideration  the 
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UPPER  BOUND  ENVELOPE 


6 msg/sec  j 
4 msg/sec  j 
XS  = 07  nCt 


LOWER  BOUNDS 


LOWER  BOUND  ENVELOPE 


May  at  Constant  Throughput  par  Nock;  Modal  with  Updatas,  Xu  - X® 


UPPER  BOUND  ENVELOPE 


LOWER  BOUND  ENVELOPE 


6 msg/sec 
4 msg/sec 
N%/2  .07  tiC. 


LOWER  BOUNDS 


Figure  4.1 1.  May  at  Constant  Throughput  par  Noda;  Modal  with  Updatai,  Xu 


limitation  of  nodal  storage.  Then  based  on  that  model,  we  study  the 
behavior  of  hierarchical  routing  as  applied  to  the  class  of  symmetrical 
networks.  We  choose  not  to  account  for  the  effect  of  updates  on  the 
line  capacity  utilization,  in  order  both  to  model  situations  where 

^updates  can  be  neglected  and,  maiiily,  to  isolate  the  consequences  of 

X 

finite  nodal  storage  on  the  network  performance. 

Similar  to  the  previous  section,  this  study  will  also  demon- 
strate a remarkable  efficiency  of  hierarchical  routing  in  large 
networks . 

4.4.1  A Loss-Queueing  Model  for  Synetrical  Networks 

4.4. 1.1  The  Model 

Again  we  consider  the  class  of  symmetrical  networks  and,  in 
addition,  a constraint  is  imposed  on  the  number  of  buffers  reserved  for 
the  store -and- forward  function.  As  a result  of  the  limited  storage, 
three  issues  arise:  the  validity  of  the  exponential  message  length 

distribution;  the  fate  of  the  rejected  messages;  and  the  sharing  of  the 
pool  of  S/F  buffers  among  the  outgoing  channels. 

With  respect  to  the  message  length,  it  is  clear  that  a maximum 
size  must  be  imposed,  as  is  always  the  case  in  practical  situations. 

As  an  example,  in  the  ARPANET  the  maximum  packet  size  is  equal  to  1008 
data  bits.  The  It§>  S/F  storage  is  divided  into  buffers  (pages),  each 
of  which  can  accomodate  a maximum  size  packet  and  cannot  be  utilized 
by  more  than  one  packet  at  a time.  As  a result,  one  might  feel  that 
the  assumption  of  exponentially  distributed  packets  should  be  replaced 
with  a constant  length  packet  assunption.  However,  measurements  on  the 


ARPANET  [KLEI  74]  have  shown  that  the  average  size  of  a data  message  is 
roughly  250  bits,  the  fact  that  the  average  message  length  is  much 
smaller  than  the  buffer  size,  and  recognizing  that  messages*  which  do 
not  fit  in  a single  buffer  occur  with  a very  small  probability  (and 
hence  can  be  neglected) , motivates  us  to  keep  the  exponential  message 
length  assunqption.  A better  approximation  would  be  to  assume  a trun- 
cated exponential  message  length  distribution,  but  this  makes  the 
analysis  much  more  complicated  and  no  closed  form  solution  has  been 
obtained  [CHU  69]. 

As  for  the  rejected  messages,  they  can  be  either  transmitted 
by  the  sending  node  (after  a time-out,  as  in  the  ARPANET)  or  considered 
as  lost  (as  with  blocked  telephone  calls).  The  retransmission  mode  is 
what  actually  prevails  in  general  S/F  networks  like  the  ARPANET,  NPL, 
etc.  However,  this  mode  introduces  strong  dependencies  among  the  sto- 
chastic behavior  of  neighboring  (and  even  more  distant)  nodes  (ZEIG  71] 
to  the  point  that  an  analysis  seems  out  of  reach.  As  a result,  we  will 
restrict  our  considerations  to  a loss  model,  in  which  case  the  depen- 
dencies between  nodes  due  to  storage  limitation  are  eliminated. 

Finally  with  respect  to  the  sharing  of  the  pool  of  S/F  buffers 
between  the  outgoing  channels,  the  S/F  function  of  a node  can  be  rep- 
resented as  in  Fig.  4.12a.  There,  we  see  that  accepted  messages  are 
first  submitted  to  the  routing  policy  and  then  conceptually  join  a pool 
of  buffers  (B  buffers) . The  routing  decision  is  assumed  to  be  fixed 
(hence  independent  of  buffer  utilization).  As  a consequence,  a node 


Recall  the  single  packet  messages  assumption. 


is  equivalent  to  R single  queueing  systems  which  share  a pool  of  B 
buffers  under  some  scheme.  A given  channel  can  only  serve  messages 
routed  to  that  channel;  hence  messages  can  be  considered  as  forming  R 
classes  of  customers,  and  those  pertaining  to  class  i are  to  be  uniquely 
served  by  the  ith  channel  of  that  node  i « 1,  2,  . ..,  R (see  Fig. 
4.12b).  Also  similar  to  the  independence  assumption  [KLEI  64],  we  will 
assume  that  nodes  (channels  in  different  nodes)  are  stochastically 
independent  and  that  the  input  streams  at  each  node  are  governed  by 
independent  Poisson  distributions. 

As  a result  of  the  above  considerations  and  assumptions  intro- 
duced along  with  them,  the  network  can  be  considered  as  a collection  of 
independent  nodes,  each  of  which  can  be  modelled  as  R m|m|1  queues 
sharing  a waiting  room  of  size  B.  Hie  traffic  offered  at  each  node  is 
governed  by  a fixed  routing  decision,  and  the  probability  of  blocking 
at  a node  is  a function  of  the  sharing  scheme  utilized.  Several  such 
schemes  are  proposed  and  studied  in  Appendix  B.  We  list  among  them: 
complete  sharing  (CS) ; sharing  with  a constraint  on  a maximum  queue 
length  (SMXQ);  sharing  with  a minimum  allocation  of  buffers  per  queue 
(SMA),  etc. 

4. 4. 1.2  Analysis 

We  first  consider  a few  remarks  before  we  proceed  with  the 
analysis.  Because  of  the  symmetry  of  our  class  of  networks,  we  assume 
that  the  fixed  routing  results  in  equal  offered  loads  on  the  channels. 
The  offered  load  X is  defined  as  the  input  rate  of  traffic  before 
acceptance  or  rejection  by  a node.  Moreover,  all  nodes  are  assumed  to 
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contain  the  same  number  of  buffers  B and  to  use  the  same  buffer  sharing 
strategy.  As  a result,  the  probability  of  blocking  (to  be  denoted  by 
PB)  is  the  same  at  all  nodes. 

Because  of  the  possiblity  of  loss  of  messages,  the  offered 
external  traffic  T is  no  longer  equal  to  the  throughput  of  the  network, 
which  we  denote  by  rs  (s  for  successful  traffic).  In  what  follows,  we 
intend  to  find  and  the  average  delay  T of  the  successful  traffic. 

Throughput  versus  load 

T is  now  referred  to  as  the  traffic  load.  Let  us  define  P 

s 

as  the  probability  that  in  a steady  state,  a message  transmitted  over 
the  network  reaches  its  destination  successfully.  Clearly 


ps  - rs/r 


(4.44) 


Let  h be  the  discrete  random  variable  representing  the  distance 
in  hops  between  any  pair  of  nodes,  as  derived  from  the  fixed\routing 
policy.  Also,  let  Pr[h  = k]  be  the  fraction  of  node-pairs  at  distance  k 
and  H(z)  the  corresponding  z-transform,  i.e.,  H(z)  = E^P^h  s 
(See  Appendix  A,  Section  A. 4 for  more  details  and  for  the  computation 
of  H(z)  for  a torus  net.) 

Because  of  the  uniform  traffic  assumption, 

ps  “ 5^  pr  [message  successful/message  travels  over  k hops]  Pr[h  = k] 

Since  nodes  are  assumed  independent,  the  probability  that  a message  is 

• 1c 

not  rejected  over  k hops  is  [1-Pg]  . Notice  that  a generated  message 
is  subject  to  rejection,  whereas  a message  reaching  its  destination 
is  always  accepted.  Thus 
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And  fro®  Eq.  (A. 25), 


£ [1  - PJ1^  fh  = k] 
k>l  B r ~ 


Hence 


Ps  - H(1  - PR) 


rs  = H(i  - PB)r 


(4.45) 


(4.46) 


Relation  between  the  load  T and  the  total  offered  internal  traffic  A 

A is  now  the  sum  of  the  offered  input  rates  to  all  the  network 
channels.  With  the  above  definition  of  A,  the  relation  A = NAX  still 
holds  true.  However  Eq.  (4.6)  (A  * nT)  is  no  longer  true  due  to  the 
possible  loss  of  messages.  A similar  approach,  as  used  for  the 
derivation  of  Eq.  (4.6)  in  [KLEI  64],  is  considered  here  to  derive  a 
relation  between  A and  T 

The  contribution  of  y , the  rate  of  traffic  from  s to  t 
(Yst  ■ Y)»  to  A is 


h.t-‘ 


£.  • vk 


1 - (1  - P„) 


The  proof  goes  as  follows:  if  h is  the  len&  h (in  hops)  of  the 

unique  path  from  s to  t,  then  the  contribution  of  y to  the  1st  hop 
(i.e.,  node  s)  is  y#t;  to  the  2nd  hop,  ygt(l  - Pg) ; and  in  general, 
to  the  k hop  it  is  Yst(l  - Pg)*  * • The  rest  of  the  proof  is  obvious. 

Recall  that  T « N(N  - l)y  and  y « y,  then 


. . r , ^ a - V ' 

A T 1 * 
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Growing  together  all  paths  of  length  k and  using  Eqs.  (A. 24)  and  (A. 25) 
we  arrive  at 


«r  1 ~ £ (1  - PJkP  [h  = k] 

B L k>l  B r ~ J 


hence 


1 - H(1  - PB) 


1 - P 


(4.47) 


B B 

The  above  relation  (and  Eq.  (4.46))  is  quite  general;  it  only  assumes 
that  all  nodes  are  independent  and  have  an  equal  probability  of  blocking 
Pg-  H(z)  can  be  determined  analytically  or  numerically  given  a partic- 
ular network  and  the  associated  fixed  routing  policy. 

Notice  that  if  Pg  * 0,  i.e.,  infinite  nodal  storage  assumption, 
Eq.  (4.47)  becomes  undefined;  however,  the  application  of  L'Hopital's 
rule  results  in  A * H'(l)r.  H'(l)  is,  in  fact,  equal  to  the  average 

network  path  length;  hence  we  are  back  to  the  expression  derived  in 
[KLEI  64]  (i.e.,  A = itf) . 


Average  delay  of  successful  traffic 

Due  to  the  symmetry  of  our  class  of  nets,  a non-rejected 
message  will  incur  the  same  delay  t at  each  hop;  therefore  the  average 
network  delay  is. 


V 


where  ng  is  the  average  path  length  of  the  successful  traffic.  Note 


that  h is  no  longer  the  average  path  length;  this  is  intuitively  due 
to  the  fact  that  messages  which  travel  on  longer  paths  are  more  likely 


to  be  rejected.  Therefore,  we  expect  ng  to  be,  in  general,  smaller 


than  h.  The  determination  of  n follows  similar  to  that  of  P . 

s s 


1 
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Let  Yjk  be  the  rate  of  successful  traffic  from  node  j to  node  k,  and 
hjk  path  length  between  the  two  nodes,  then  obviously 


_ h 

*jk  - <>  - p»>  % 


Also,  the  sum  of  Y*k  is  what  we  have  defined  as  I*  . Then,  from  the 
definition  of  the  average  path  length  (Eq.  (A. 3)), 

£ E yJj. 


n 


*-K  ~ ■ r ? ? “ - V*  Vjk 


s EE/ 


j k 


jk 


s j k 


Recall  that  Yjk  = Y,  and  T = N(N  - l)y;  thus 


’ 17  ? ? (1  - VV  Nfir^TT 


Groining  together  all  paths  of  length  k,  we  arrive  at 

",  * 17  S « - PB)kkPr1!!  - 

From  the  definition  of  H(z)  (Eq.  (A. 25)  and  Eqs.  (4.44) , (4.46)) , we 
find 


1 - P 


n 


B 


H’(l  - PB) 


— H-(i  - pB>  - a - pB) 


(4.48) 


where  H*(z)  is  the  derivative  of  H(z)  with  respect  to  z. 


Note  that  if  PR  « 0,  then  n.  » h (recall  that  H»(l)  = h and 
H(l)  - 1). 

If  we  let  Pg  1 and  apply  l'Hopital's  rule  to  the  above 
equation  (recall  that  H(0)  « 0),  we  arrive  at  n « 1.  This  result 
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1 
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■45 


indicates  that  at  the  limit  (Pg  1)  only  1-hop  traffic  may  still  be 


successful. 


Probability  of  Blocking 


As  noticed  earlier,  a node  may  be  modelled  by  R M|M|l  queueing 


systems  with  a shared  finite  waiting  room  of  size  B.  Each  server  is 


offered  an  input  rate  A (A  = A/NA,  Eq.  (4.47)),  and  has  a service  rate 


equal  to  yC.  Among  the  sharing  schemes  studied  in  Appendix  B,  complete 
sharing  is  optimal  when  p is  small.  Below,  we  show  that  the 


maximum  throughput  T is  obtained  for  a load,  say  T , which  is  such 

s m 


that  P < 1;  moreover  in  order  to  maintain  a good  network  performance 


some  control  should  be  introduced  in  order  to  keep  the  offered  load  at 


a level  less  than  or  equal  to  T . As  a result  it  seems  reasonable  to 

m 


choose  a complete  sharing  (CS)  scheme.  However  if  a retransmission 


mode  is  used  or  if  the  channels  receive  veiy  unbalanced  traffic  loads 


then  a different  scheme  may  be  necessary. 


The  analysis  of'  the  CS  scheme  in  Appendix  B leads  to  an 


expression  of  Pg  in  terms  of  A/yC,  Eq.  (B.33),  which,  combined  with 
Eq-  (4.47)  results  in  the  system  of  equations  below. 


1 - H[1  - PB]  r 


(B ; ; ; ‘HA) 

I W 


(4.49) 


Also  the  average  delay  per  channel,  t,  for  successful  traffic  is  given 
by  Eq.  (B.37)  hence 


T = 


(4.50) 


The  above  system  of  equations  (4.49a-b),  may  be  solved  numer- 
ically. A numerical  algorithm  will  be  presented  below.  Once  we  know  X 
and  Pg,  we  can  determine  the  performance  of  the  net  in  terms  of 
throughput  rg  and  delay  T.  Before  we  proceed,  let  us  derive  the  limit- 
ing throughput  of  the  network  obtained  with  an  infinite  traffic  load. 


Limiting  throughput 

We  claim  that 

JiDlit  rs  = B ♦ k - 1 H'WNApC  (4. SI) 

Proof: 

From  the  determination  of  A we  see  that  X > T/RN  (i.e.,  X is 
greater  than  1/R  the  traffic  generated  at  the  node  itself),  hence 
T -►  oo  ->  X -*•  » . Also  from  Eq.  (4.49b)  it  is  clear  that 

PB  • 1 - FT  I - i T*  0(1/x>  (4.52) 

where  0(1/X)  is  such  that  limit  X0(1/X)  = 0.  Thus,  limit  P =1. 

X -*•  » x-»  B 

Now  from  Eq.  (4.49a)  we  see  that  Pg  * 1 ■=>  X = T/NA.  As  a result. 
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if  itifff  *“*i  in 


- ■ -|| -* i ] ■ DfnTittri* 


limit  X -4  = 1 

r -*■  oo  1 

Also  from  Eq.  (4.52) 

limit  i (1  - P„)  - g ■ i PC 

I -►  00 

Now  from  Eq.  (4.46) 

rs  = TH(1  - PB)  = (1  - PB)rH'(0)  ♦ rod  - pb) 

Then,  combining  the  above  facts,  we  arrive  at  Eq.  (4.51). 

This  limiting  result  has  the  following  simple  interpretation. 
First,  note  that  H'(0)  = Pr[h  = 1]  is  the  fraction  of  pairs  of  nodes 
at  distance  one  (i.e.,  neighboring  nodes),  and  (ByC/(B  ♦ R - 1)  is  the 
limiting  throughput  of  any  channel  of  the  R M|M|l  system  of  queues 
(see  Eq.  (B.39)).  As  a result,  the  limiting  throughput  represents  the 

fraction  of  successful  traffic  which  has  to  travel  over  a single  hop. 

Hie  other  fraction  (finite)  of  initially  successful  traffic  has  to 
travel  over  at  least  another  hop;  in  trying  to  do  so,  it  will  compete 
with  an  infinite  amount  of  traffic  generated  at  the  next  node,  and  thus 
it  will  be  rejected.  This  checks  with  the  previous  result:  Pg-+-  l=ons+  1. 

Algorithm  for  the  solution  of  Eq.  (4.49) 

Let  us  study  the  behavior  of  the  two  equations  (4.49  a - b) 
i.  Variation  of  X versus  Pg,  Eq.  (4.49a).  Differentiating 
Eq.  (4.49a)  with  respect  the  Pg,  we  find 
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From  the  definition  of  H(z),  Eq.  (A. 25) 


H'(z)  = £ kzk_1P  [h  = k] 
k>l  r ~ 


Let  z = 1 - P , the  numerator  of  the  2n<*  fraction  becomes 


Y = (1  - z)H'(z)  - 1 + H(z) 


Y = £ [zk  - 1 + (1  - z)kzk_1]  P [h 
k>l 


r~ 


k] 


Let  be  the  quantity  in  brackets 


dX, 

dT=  k(k  - l)zk-Z(l  - z) 


Hence  for  0 £ z £ 1»  dX^/dz  >.  0,  thus  X^  is  an  increasing  function  of 
z and  finally  Y is  an  increasing  function  of  z.  When  z varies  from 
0 to  1,  i.e.,  when  Pg  varies  from  1 to  0,  Y varies  from  H'(0)-  l to  0. 

Since  H* (0)  = Fr[h  - 1],  then  H'(0)  - 1 < 0 and  Y < 0 ¥ ze  [0.1 j. 
Therefore,  A is  a decreasing  function  of  Pg  (see  Fig.  4.13)  and  from 
previous  remarks:  PR  = 1 ->  \ = p/NA;  Pg  = 0 ■=»>  A * hf/NA. 


ii.  Variation  of  Pg  with  respect  to  A , Eq.  (4.49b) 


It  can  be  easily  shown  that  the  derivative  of  Pfi  with  respect 
to  A is  always  positive.  Thus  Pfi  is  an  increasing  function  of  A.  Also 
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r/NA  h t/na  x 


Figure  4.13.  Solution  of  Eq.  (4.49). 


iii.  Algorithm. 


Pig*  (4.13)  shows  that  there  is  a unique  solution  to  Eq. 
(4.49).  The  numerical  evaluation  is  straightforward,  and  can  use  any 
converging  iteration  procedure.  Namely,  let  us  consider  the  sequences 


{A,  } and  {P_}  such  that. 


xi  ’ 


X2  ’ «PB>  ' 

H - 


where  the  A,  *s  are  computed  from  Eq.  (4.49a)  and  the  P^'s  from  Eq. 

(4.49b).  Obviously,  A^  = limit  A.  is  the  solution  of  the  system  of 

k + * 

equations.  In  practice,  we  stop  the  search  when  l\  “ Afc  J <_  6 ; 

where  6 is  a certain  tolerance. 

We  are  now  ready  to  consider  a numerical  example. 


4. 4. 1.3  Application  of  the  Loss  Model  to  Torus  Networks 


Let  us  consider  a torus  (i.e.,  R = 4)  operated  with  a fixed 
shortest  path  routing  whose  z-transform  is  given  by  Eq.  (A. 41)  (where 
P - /n).  Of  interest  is  the  study  of  the  behavior  of  the  successful 
traffic  r with  respect  to  the  load  T,  as  well  as  the  behavior  of  the 
delay  T with  respect  to  T or 

Numerical  results  are  shown  in  Figs.  4.14  - 4.17.  Those 
results  were  obtained  for  N * 121,  6 = 10~4,  and  pC  = 20  msg/sec. 

More  precisely,  the  graphs  show  the  normalized  traffic  and 
delay.  The  normalization  is  based  on  Eq.  (4.19)  which  defines  the 
utilization  of  the  net,  and  on  Eq.  (4.18).  Thus  the  curves  show  the 
variables 
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Figs.  4.14  and  4.15  show  (linear  and  semi-log  abscissa)  that  as  T 
increases,  increases  to  a maximum  value  and  then  decreases  to  its 
limiting  value  in  Eq.  (4.51).  These  results  are  similar  to  that  of 
a contention  system  [TOBA  74],  [LAM  74],  except  that  the  non- 
retransmission (loss)  of  rejected  messages  eliminates  the  possibility 
of  unstable  states. 

Note  that  if  B = 00  then  is  equal  to  T for  p varying  from 
0 to  1.  For  p ^ 1 a steady  state  solution  does  not  exist.  This  is  no 
longer  true  for  a finite  buffer  size.  However,  with  limited  storage, 
as  p increases  beyond  1,  the  throughput  decreases  quite  a bit  and  thus 
more  messages  are  being  lost.  Another  effect  of  finite  storage  is 
reflected  in  the  behavior  of  the  average  delay  T which  asymptotically 
reaches  a constant  value  as  p goes  to  infinity  (see  Fig.  4.16).  Since 
for  p -*■  00  (i.e.,  T ->■  °°)  only  1-hop  traffic  may  be  successful  (see 

Eq.  (4.51)),  then  the  asymptotic  value  of  T corresponds  to  the  delay 
on  one  hop  (i.e.,  at  one  node)  under  the  condition  of  an  infinite  input 
rate.  From  Eq.  (B.39),  that  value  for  one  node  is:  limit  yCt  = 

. p -*■  00 

B/(B  + R - 1);  thus  limit  yCT/h  = B/[(B  + R - l)h]. 

p ■+■  OO 

Fig.  4.17  illustrates  the  behavior  of  T versus  I*  . Note  that 
to  a given  Ts  correspond  2 points  on  the  curve.  This  is  due  to  the 
fact  that  there  are  two  possible  values  of  T which  lead  to  a specific 
value  of  rg  (see  Fig.  4.14). 

With  regard  to  the  number  of  buffers  B,  the  plots  show  initial 
substantial  improvements  in  maximum  throughput  as  B increases,  and 
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costlier  ones  when  we  reach  values  of  pg  approximately  between  0.8  and 
0.9.  This  phenomenon  is  better  shown  in  Fig.  4.18  where  the  probability 
of  loss,  1 - P , is  plotted  versus  the  normalized  load  for  different 
values  of  B.  Finally,  Fig.  4.19  shows  the  behavior  of  ng  normalized  by 
h,  with  respect  to  1 - P . 

In  summary,  we  are  now  able  to  evaluate  the  importance  of  buf- 
fer storage,  and  as  expected,  small  values  of  B can  degrade  the  network 
performance  quite  a bit.  For  the  example  above,  B = 20  reduces  the 
performance  to  roughly  0.68  that  which  could  otherwise  be  obtained  with 
an  infinite  number  of  buffers. 

4.4.2  Performance  Evaluation  of  Hierarchical  Routing 

The  above  model  and  Assumption  4.1  will  now  allow  us  to  study 
the  behavior  of  hierarchical  routing  for  the  class  of  symmetrical  nets. 
The  scaling  schemes  need  further  specification  as  to  the  scaling  of  the 
nodal  storage  (B) . 

4.4. 2.1  A Buffer  Scaling  Scheme 

As  seen  in  the  conclusions  above,  the  effect  of  B is  naturally 

characterized  by  the  maximum  throughput  T^,  or  using  our  normalized 

notations,  it  is  characterized  by  the  maximum  of  p * hT  /yO<A.  Under 

s s 

the  conditions  of  Section  4.1.2,  the  primal  scaling  scheme  maintained 
p constant  as  N varied.  It  is  now  natural  to  attempt  to  keep  maximum 

Ps  constant.  With  this  objective  in  mind,  let  us  observe  the  effect  of 

storage  limitation  first,  in  a single  node  situation,  then  in  a network 

environment.  From  Eq.  (4.49b)  we  see  that  PR  depends  only  on  A/pC  and 

B.  Thus  a scaling  which  maintains  A/pC  and  B constant  will  result  in 
a constant  PR.  Recall  that  under  the  conditions  of  Section  4.1.2, 
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X = hr/NA;  hence  X/pC  is  just  our  previous  p that  the  scaling  maintained 
constant. 

In  a network  environment  keeping  PR  constant  will  still  result 
in  a smaller  probability  of  success  Pg  as  the  network  grows.  This  comes 
about  because  of  the  increase  in  network  path  length  (aNv)  which  results 
from  a larger  N.  It  is  then  necessary  to  increase  B with  N,  in  order 
to  maintain  p constant. 

An  ad-hcc  (heuristic)  scaling  scheme  of  B has  been  devised 
which,  as  we  will  see,  satisfies  our  needs  to  a large  extent.  Such  a 
scheme  is 

[b  = BQ  In  hj  (4.53) 

This  scheme  has  been  tested  on  the  torus  nets  as  shown  in  Fig..  4.20 
where  pg  is  plotted  versus  p for  a set  of  values  of  N and  two 
values  of  BQ.  We  notice  that  the  maxima  of  pg  occur  for  roughly  the 
same  value  of  p.  The  curves  (except  for  N = 25)  are  also  fairly  close 
to  each  other  up  to  and  somewhat  beyond  the  maxima.  Since,  in  fact, 
networks  must  be  operated  in  that  range  and  since  we  are  dealing  with 
large  networks,  this  scaling  scheme  appears  to  be  fairly  satisfactory. 

The  aforementioned  figures  showed  the  overall  behavior  of  P 

s 


versus  p for  a set  of  values  of  N.  If  we  restrict  our  ovservations  to 
the  maximum  throughput  pg,  we  obtain  the  curves  shown  in  Fig.  4.21. 
Those  curves  were  obtained  using  the  combined  scaling  scheme 
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and  for  a maxima,  Ps  d.tarminad  .ith  or  mithout  a constraint  on  th. 

average  delay  T.  The  maximum  is  computed  numerically  using  a 

Fibonacci  search  [ZANG  69].  Ihe  probability  of  success  P$  the 

normalized  delay  pCT/h,  and  the  normalized  average  path  length  n$/h 

at  those  maxima  of  pg,  are  shown  respectively  in  Figs.  4.22  and  4.23. 

A few  remarks  emerge  from  the  observations  of  those  graphs: 

i.  The  curves  of  max  p with  no  constraint  on  T,  and  that 

s 

of  the  corresponding  T,  show  again  that  our  scaling  scheme  is  quite 
acceptable,  especially  when  N > 1000.  For  that  range  of  N,  the  maxima 
of  Ps  are  between  0.7  and  0.8. 


ii.  The  fact  that  max  P with  the  constraint  T < 1 becomes 

s — 

constant  (=  .74)  for  N roughly  greater  than  10^  can  be  explained  as 
follows.  Figs.  4.22  and  4.23  show  that  the  corresponding  Pg  and  ns/h 
are  equal  to  one  for  the  same  range  of  N.  This  means  that  at  such  a 
load,  B is  large  enough  to  give  a negligible  loss;  hence  it  is  safe  to 
use  the  earlier  model  based  on  the  infinite  nodal  storage  assumption 

see  Section  4.1.2).  That  model  predicts,  with  C = hCQ  and  T = TQ  = 1, 

~i  Hr 

a value  of  T/NA  = pC_  - T . Thus  we  expect  max  p — = 

00  s yC  NA 

= .74,  which  checks  with  the  value  found  in  the  plots. 

A final  question  arises  as  to  the  sensitivity  of  the  above 
results  to  the  scaling  of  C.  For  that  purpose,  let  us  prove  the  fol- 
lowing proposition. 

Consider  a symmetrical  net  of  size  N,  which  is  such  that  all 
channels  are  of  capacity  Cj  or  Cj.  Then  network  loads  proportional  to 
the  capacity  assigned,  result,  for  a fixed  B,  in  throughputs  propor- 
tional to  the  capacities,  i.e.. 
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(4.55) 


where  I\,  r*  and  T.  are  the  load,  throughput  and  delay  with  assignment 
C.  (i  - 1,  2) 

Proof 

Let  Aj  and  P^  be  the  unique  solution  of  Eq.  (4.49)  for 

T = Tj  and  C = Cj.  It  is  obvious  that  A 2 = AjP^/pCj  and  Pg2^  ■ Pg1-* 

is  the  unique  solution  of  Eq.  (4.49)  for  r * T2  and  C = C2«  This  fact 

combined  with  Eqs.  (4.46)  and  (4.50)  proves  the  above  proposition. 

As  a result,  the  normalized  notation  (p,  ps)  is  not  sensitive 

to  the  particular  capacity  assignment  for  a given  N and  B,  i.e., 

P1  * P2  pl  ’ ps  • 

Hie  above  property  implies  that  the  same  curves  as  in  Fig.  4.20 
can  be  obtained  with  any  capacity  assignment  for  a given  N and  B. 
Moreover,  the  maxima  of  pg  are  also  insensitive  to  the  particular  choice 
of  capacity  (recall  that  all  channel  capacities  are  equal)  at  a given 
N and  B.  This  implies  that  all  the  points  in  the  curves  in  Figs.  4.21  - 
4.23  depend  only  on  N and  the  buffer  scaling. 

With  the  capacity  scaling  C ■ hCQ  , Pg  becomes 

ps  * ^ Nl  (4‘56) 

which  indicates  that  if  the  scaling  of  B keeps  pg  constant,  then  the 
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scaling  of  C will  keep  the  throughput  per  node  constant,  and  that  ful- 
fills our  objective. 


4. 4. 2. 2 Behavior  of  the  Hierarchical  Routin 


Recall  that  Assumption  4.1  leads  us  to  consider  a fixed  routing 
to  model  an  adaptive  hierarchical  routing.  Provided  we  know  the  distri- 
bution of  the  path  length  of  the  equivalent  routing,  we  can  use  the  loss 
model  and  scaling  scheme  to  predict  the  behavior  of  such  routing  schemes 
as  m and  N vary.  Once  again,  we  use  the  bounds  of  Chapter  3 to  charac- 
terize the  message  path  lengths  which  result  from  a hierarchical  routing 


Distribution  of  Path  Length 

Since  the  distribution  of  path  length  deals  with  paths  on  a 
node-pair  basis,  we  can  no  longer  use  the  bound  E,  which  was  only  valid 
for  the  average  distance.  Fortunately  Lemma  3.2  (see  Eq.  (3.29))  pro- 
vides us  with  the  more  general  bound  on  individual  paths  (to  be  denoted 


by  A) 


hst  • hst  i 4 ■; £ \ es 

k*l 


However  this  bound,  always  true  for  CER,  is  only  valid  with  OBR  if  s,t 
belong  to  a lower  level  cluster  than  Ca  (see  Eq.  (3.31)).  Ihe  fact  that 
first,  A is  a very  generous  bound,  and  second  that  we  expect  OBR  to 
behave  better  than  CER  makes  us  feel  confident  to  use  A as  an  approxi- 
mation on  paths  for  the  OBR  scheme.  Note  that  a rigorous  but  extremely 

c A 

generous  bound,  may  be  obtained  from  Leman  3.1  (h  < £ d.). 

st  k«l  * 

From  Eq.  (3.46)  and  for  our  class  of  networks,  A is  simply  equal  to  hF 
(see  Eqs.  (3.46)  and  (3.47));  and  for  an  optimal  clustering  of  degree  m 


(4.57) 
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Note  again  that  ■ • 1 ™»  A ■ 0. 

A '•best  case"  and  a "worst  case"  distribution  can  now  be 

c c 

defined  if  we  assume  respectively  that  hgt  = hgt  and  hgt  = hgt  + A. 

To  the  "best  case"  distribution  corresponds  the  z-transform 
H(z)  of  the  shortest  paths  in  the  network. 

The  z-transform  Hc(z)  of  the  worst  case  distribution  can  be 
expressed  in  terms  of  H(z)  and  A,  as  follows 

Hc(z)  = z^H(  z)  (4.58) 

To  prove  the  above  equation,  let  h£  be  the  deterministic  random  variable 
representing  the  path  length,  then 


P (h  * k]  = P [h  * k - A]  ¥ k > A 
r ~c  r 1 — 

Thus 

H (z)  - Z zkP  th  - *1  = zA  £ z'PAh  * i] 
c k>A  r c i >0  r ~ 

which  proves  Eq.  (4.58). 

The  above  remark  that  m = 1 —>  A = 0 implies  also  that 

m » 1 — > H (z)  = H(z).  This  fortunate  property  preserves  the  conti- 
c 

nuity  of  hierarchical  routings. 


Buffer  Assignment  and  Feasibility 

Recall  that  in  this  study  we  intend  to  account  for  the  storage 
utilized  by  the  Touting  tables.  The  size  of  such  a storage  is  a linear 
function  of  the  table  length  and  counted  in  nustoer  of  buffers  it  is 
equal  to 
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For  a hierarchical  routing  to  be  feasible,  B must  be  greater  than  or 
equal  to  one.  Observing  Eq.  (4.60)  we  may  conclude  that: 

i.  For  a fixed  m,  the  routing  becomes  infeasible  for  networks 

of  size  larger  than  a critical  number  N . N is  the  solution  of  B * 0 

c c 

in  Eq.  (4.60)  and  it  is,  obviously,  an  increasing  function  of  m. 

ii.  For  very  large  networks,  and  under  the  condition 

Bqv  _>  t-2^,  only  a hierarchical  routing  operating  with  a global  minimum 
table  length  is  feasible,  i.e.,  m * m*  = In  N;  hence  l * mN*^m  = e In  N. 

In  summary,  as  N gets  larger,  it  becomes  imperative  to  move 
toward  more  clustering,  eventually  reaching  a globally  minimum  table 
length.  The  decision  to  use  a higher  degree  of  clustering  m should  be 
weighed  against  the  degradation  incurred  by  the  corresponding  increase 
in  network  path  length.  This  phenomenon  is  illustrated  in  the  next 
section. 


Numerical  Application 


From  the  above  considerations,  the  application  of  the  loss 


IM 
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model  to  a network  operated  with  a hierarchical  routing  results  in  the 
evaluation  of  the  worst  and  best  case  performances. 

The  worst  case  performance  is  characterized  by 

rs  ■ Hc(1  • PB)r 

The  probability  of  blocking  PR  is  the  solution  of  Eq.  (4.49)  where  H 

is  replaced  by  H . We  note  that  because  H (z)  is  similar  to  H(z)  the 

c 

iterative  algorithm  devised  to  solve  Eq.  (4.49)  is  still  valid,  except 
that  the  initial  value  of  the  sequence  {X^}  is  now 


limit 

v° 


1 - M1  - V r r 

£ 2 i uifn 

Pg  NA  cu;  NA 


with  H • ( 1)  = H'(l)  + A * h ♦ A.  With  respect  to  the  delay  T,  let  nc 
c s 

be  the  average  path  length  of  the  successful  traffic,  replacing  H.(z)  by 
H (z)  in  Eq.  (4.48),  we  arrive  at  n c = n + A;  therefore,  T = (n  ♦ A)t 

0 5 S 5 

instead  of  Eq.  (4.50). 

The  "best  case"  performance  is  obtained  by  setting  A *=  0. 

Again  we  use  the  set  of  values  of  a,  b,  c,  v as  derived  for 
the  torus  nets,  Eq.  (3.50),  and  the  z-transform  H(z)  as  given  in  Eq. 

(A. 31).  The  scaling  is  specified  by  the  usual  C ■ hCQ  and  Eq.  (4.59) 
Also  £2  is  chosen  to  be  equal  to  1/64  (this  value  is  motivated  by  the 
ARPANET  where  a buffer  holds  up  to  64  words.) 

We  first  evaluate  the  behavior  of  an  MIR  as  applied  to  a 
specific  network  size  N ■ 1681.  Fig.  4.24  shows  the  plots  of  the  lower 
bound  on  ps,  the  normalized  thoughput,  versus  p,  the  normalized  load, 
and  this  for  a set  of  values  of  m.  In  that  exa^ile,  BQ  is  chosen 
equal  to  9;  with  that  value  the  number  of  S/F  buffers  at  m ■ 1 
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(no  clustering)  is  equal  to  one.  We  observe  that  the  best  performance 
(lower  bound)  is  obtained  for  m = 1.25  from  among  the  set  of  m's 
selected  here.  With  m = 1.25  the  maximum  pg  improves  dramatically  from 
Ps  * 0*02  with  m * 1,  to  a minimum  of  0.58.  We  also  notice  that  the 
performance  improves  as  m varies  from  1 to  1.25  and  then  it  deteriorates 
for  m > 1.25.  Moreover,  the  curves  with  a smaller  m exhibit  a slower 
decay.  This  fact  may  be  explained  as  follows:  from  Eq.  (4.51),  the 

limiting  throughput  is 

b"Y  R - 1 h;(0)namc 

but 

H^(z)  = AzA_1H(z)  ♦ z^H* (z) 

hence  z = 0 — > H^(0)  = 0 (for  A > 1,  which  is  usually  the  case,  see 
Eq.  (4.57)),  and  also  the  larger  A is,  the  faster  is  the  convergence  of 
H^(0)  to  zero.  Furthermore,  for  a fixed  N,  A is  an  increasing  function 
of  m. 

If  we  limit  our  considerations  to  an  operational  range  of  p 
(i.e.,  p is  only  allowed  to  vary  from  0 up  to  a value  slightly  larger 
than  the  one  producing  the  maximum  p.s,  roughly  0 <_  p £1),  then  the 
value  of  m which  leads  to  the  maximum  pg  also  leads  to  the  best  per- 
formance over  that  entire  range. 

As  a consequence  of  the  above  considerations,  from  now  on  we 
will  restrict  our  observations  to  the  behavior  of  the  maximum  P 

s 

(lower  or  upper  bound)  with  respect  to  N and  for  a set  of  values  of  m. 
Furthermore  the  maximum  pg  will  be  determined  with  or  without  a con- 
straint on  the  average  delay  T. 
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Fig.  4. 25  shows  such  a behavior  of  max  pg,  with  no  constraint 
on  T.  Figs.  4.26  - 4.29  show  respectively  the  corresponding  average 
delay  T,  the  probability  of  success  Pg,  the  number  of  S/F  buffers  B 
and  finally  the  corresponding  relative  bound  on  the  increase  in  path 
length  A/h. 


Figs.  4.30  - 4.32  show  similar  plots  for  max  Pg  with  a con- 
straint on  the  maximum  delay  for  T and  Pg.  A few  remarks  emerge  from 
the  observation  of  those  figures.  Those  remarks  are,  in  general, 
quite  similar  to  the  ones  stated  at  the  end  of  Section  4. 3.2.2,  namely 
with  regard  to  the  optimal  degree  of  clustering  m for  a given  N, 
and  the  feasibility  and  viability  of  hierarchical  routing.  Before  we 
proceed,  let  us  notice  that  the  non-smoothness  of  the  curves  is  due  to 
the  discrete  changes  of  B (see  Eq.  (4.59)).  This  fact  is  more  accen- 
tuated for  the  smaller  values  of  N where  B is  small,  and  consequently 
a change  of  one  unit  is  relatively  noticeable.  Round-off  errors,  as 
well  as  errors  due  to  our  numerical  algorithms  for  finding  Pg  and 
especially  max  ps  (Fibonacci  search),  may  also  contribute  to  the  non- 
smoothness  of  the  curves. 

Hierarchical  routing  with  an  appropriate  degree  of  clustering, 
n,  guarantees  a behavior  of  the  max  ps  (with  respect  to  N)  to  lie 
between  the  upper  and  lower  bound  envelopes.  It  is  quite  remarkable 
that  the  lower  bound  envelopes  (with  or  without  a constraint  on  T)  remain 
relatively  flat  (around  0.6),  for  N beyond  one  hundred.  Moreover  the 
upper  bound  envelopes  are  very  close  to  the  curves  obtained  in  Fig. 

4.21;  this  means  that  at  the  point  (N,  m)  corresponding  to  those 
envelopes,  the  storage  required  by  the  updates  is  relatively  negligible. 
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Figure  4.30.  Maximum  Throughput  with  a Dal  ay  Constraint,  Performance  of  tha  MHR; 
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(This  fact  is  also  illustrated  in  Fig.  4.28).  As  a consequence,  the 
gap  existing  between  the  lower  and  upper  bound  envelopes  is  mainly 
caused  by  the  increase  in  the  path  length  A. 

Finally,  let  us  notice  that  the  performance  of  a non- 
hierarchical  routing  (m  * 1)  deteriorates  very  sharply  for  values  of 
N around  1000,  and  that  for  N greater  than  roughly  250,  hierarchical 
routing  clearly  becomes  superior. 

4*5  A Queueing  Model  with  Updates  and  Storage  Limitation 

In  this  section,  we  intend  to  put  our  previous  efforts  to- 
gether in  order  to  devise  a model  whereby  both  line  capacities  and 
nodal  storage  used  by  the  routing  are  accounted  for. 

The  R M|M|l  single  node  model  with  a finite  number  of  buffers 
B must  now  be  modified  in  order  to  accomodate  for  the  updates.  Because  of 
the  results  in  Section  4.3.1,  a channel  can  be  modeled  by  a HOL  priority 
queue  (m|m|i,  d|d|1).  This  is,  however,  a major  obstacle  in  an  analyt- 
ical solution.  A more  careful  observation  of  the  analysis  of  the  HOL 
system  (Eq.  (4.33))  shows  that  the  effect  of  the  updates  is  primarily 

to  reduce  the  line  capacity  available  for  data  traffic  from  C to 

2 

(1  - PU)C.  The  secondary  effect  is  the  added  term  \u/2(yuC)  in  the 
numerator  of  Eq.  (4.33);  it  is  this  effect  that  we  will  neglect  in  the 
sequel.  Moreover,  we  will  assume  that  the  handling  of  updates  utilizes 
some  storage  (working  storage)  other  than  the  S/F  area. 

% 

As  a result  of  the  above  considerations,  our  study  here  is 
reduced  to  the  one  performed  in  Section  4. 4. 2. 2 where  C is  to  be 
replaced  by  (1  - p)C. 
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The  results  of  a numerical  exanple  are  shown  in  Fig.  4.33. 

The  same  environment  which  generated  Figs.  4.25  - 4.32  is  assumed  here, 
except  that  C is  now 


Pc  = hyC  - — y 
u 


The  rate  Au  selected  corresponds  to  our  earlier  coop romi sing  choice, 
i.e.,  Ay  = jN1/4A°  , where  aJ  = 0.14  yCQ.  Also  recall  Eq.  (4.35), 

_y_  - _L  njj1/™ 

In  addition,  y is  set  to  one. 

Fig.  4.33  shows  only  the  worst  case  performance1  of  the 
hierarchical  routing.  The  results  are  quite  comparable  to  those  in  Figs. 
4.25  and  4.30.  The  effect  of  updates  can  be  seen  in  the  drop  of  the 
minimum  normalized  throughput  by  roughly  0.05,  except  for  very  large 
N's  where  the  drop  becomes  very  small.  In  addition,  hierarchical  rout- 
ing becomes  superior  to  non-hierarchical  routing  fbr  N at  around  180 
instead  of  the  previous  250. 

By  co^aring  Figs.  4.7,  4.25  and  4.33,  we  note  that  in  this 
particular  example  the  network  is  more  storage-bound  than  capacity- 


bound. 


Finally,  the  fairly  flat  shape  of  the  lower  bound  envelope  of 


Fig.  4.33  shows  that  the  effect  of  adaptive  routing  in  terms  of  nodal 
storage  and  capacity  requirements  has  been  gradually  reduced  (as  N grows) 
so  as  to  keep  a smooth  network  behavior. 


The  upper  bound  envelope  obtained  in  Fig.  4.25  is  also  valid  with  this 
application. 
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4.6 


Conclusion 


In  this  chapter  we  were  able  to  demonstrate  that  for  a class 
of  symmetrical  and  distributed  networks: 

i.  In  an  ideal  situation  of  sufficient  storage  and  line 
capacity,  the  performance  of  a non-hierarchical  routing  is,  in  general, 
better  than  the  one  obtained  with  a hierarchical  routing,  except  at  the 
limit  of  very  large  nets  where  they  become  quite  comparable,  even  for 
enormous  table  reductions. 

ii.  With  a more  realistic  situation  and  with  reasonable 
assumptions  on  network  growth,  the  hierarchical  routing  becomes  not 
only  a necessity  for  large  nets,  but  it  also  preserves  a remarkably 
good  network  performance  for  a phenomenal  range  of  the  network  size  N. 

The  particular  numerical  examples  studied  in  this  chapter 
showed  that  the  transition  point  where  hierarchical  routing  becomes 
surely  (because  of  lower  bounds)  better  than  a non-hierarchical  one 
occurs  for  N between  100  and  200. 
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CHAPTER  5 


CLUSTERING  ISSUES  AND  SIMULATION  OF  THE  WR  SCHEMES 

In  this  chapter  we  address  the  following  two  issues: 

(i)  The  assignment  of  nodes  to  clusters,  clusters  to  super- 
clusters, etc.,  given  an  arbitrary  network  and  a clustering  structure, 
(i.e. , n,  m) . 

(ii)  The  evaluation  of  the  hierarchical  routing  schemes  as 
applied  to  more  general  networks  such  as  the  ARPANET. 

With  respect  to  (i) , first  we  define  some  nearness  measures 
among  the  network  nodes.  Then,  based  on  those  measures,  we  choose  (and 
slightly  modify)  the  "Complete  Linkage"  (CL)  technique  as  a solution  for 
the  clustering  problem. 

With  respect  to  (ii) , a simulation  program  has  been  developed 
in  order  to  evaluate  the  performance  of  the  MIR  schemes  (OBR,  CER)  under 
some  fairly  general  operational  environment. 

5.1  Clustering  Issues 

The  clustering  problem  is  generally  defined  as  one  of  finding 
natural  groupings  in  a set  of  data,  points,  objects,  etc.  Numerous 
clustering  techniques  or  procedures  have  been  documented  in  the  litera- 
ture. We  mention  in  particular  the  comprehensive  book  of  Anderberg 
[ANDE  73]  and  other  publications:  [DUDA  73],  [KERN  70]  and  the  bibliog- 
raphies therein. 

l 

In  our  context  of  computer  networks,  we  need  to  define  what 
we  mean  by  a natural  grouping.  In  what  sense  are  we  to  say  that  certain 
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nodes  are  more  likely  to  belong  to  the  same  cluster  than  others.  This 
question  actually  involves  two  separate  issues:  how  one  should  measure 

the  similarity  (nearness)  between  two  nodes,  and  how  one  should  carry 
out  and  evaluate  the  partitioning  of  a set  of  nodes  into  clusters.  In 
this  section,  we  address  these  issues  based  on  some  observations  and 
previous  conclusions  related  to  what  we  consider  as  a good  clustering. 


5.1.1  Characterization  of  a "Good"  Clustering 

Let  us  temporarily  restrict  our  considerations  to  a 2-level 
clustering,  i.e.,  the  partitioning  of  the  set  of  nodes  into  disjoint 
subsets  (clusters).  Recall  that  with  a hierarchical  routing,  routing 
tables  contain  one  entry  per  node  for  nodes  in  the  same  cluster  and 
one  entry  per  cluster.  Also,  traffic  between  nodes  in  the  same  cluster 
must  follow  paths  internal  to  that  cluster.  Furthermore,  a given  node 
must  send  all  its  traffic  to  a given  cluster  on  the  same  path  up  to  that 
cluster.  The  above  rules  lead  us  to  distinguish  two  cases  with  regard 
to  the  internal  and  external  properties  a cluster  should  exhibit. 

(a)  Internal  Characteristics.  The  internal  behavior  of  a 
cluster  should  take  advantage  of  the  complete  routing  information  at 
each  node.  As  a result,  cluster  subnets  must  include  most  of  the  "best" 
paths  between  nodes  in  that  cluster.  More  importantly,  since  internal 
traffic  is  confined  to  paths  internal  to  the  cluster,  the  cluster  subnets 
must  contain  the  shortest  paths  between  its  nodes.  This  last  property 
was  assumed  to  be  true  in  the  derivation  of  some  of  the  earlier  bounds 
on  the  increase  in  the  network  path  length  (see  Chapter  3) . 


To  take  further  advantage  of  the  complete  routing  information, 
in  case  of  non-uniform  traffic  conditions  (i.e.,  Y-.'s  are  not  neces- 
sarily  equal) , clusters  should  be  composed  of  nodes  with  higher 
intertraffic  rates. 

(b)  External  Characteristics.  Due  to  the  reduction  of 
"external"  routing  information,  a cluster,  as  seen  from  the  outside,  is 
equivalent  to  a single  "super-node."  An  increase  in  the  network  path 
length  has  occured  (Chapter  3)  because  of  messages  entering  a cluster 
from  a non-optimal  entry  point  (exchange  node).  The  single  most  impor- 
tant variable  which  affects  this  increase  is  the  diameter  of  a cluster, 
or  more  precisely,  it  is  the  distance  between  the  entry  points  of  a 
cluster.  As  a result,  clusters  should  be  chosen  so  as  to  correspond  to 
highly  connected  sets  of  nodes  which  thus  lead  to  a small  diameter  and, 
moreover,  to  a small  average  internal  path  length.  In  case  of  low- 
connected  nets,  long  chain  clusters  must  be  avoided. 

Under  non-uniform  traffic  conditions,  clusters  should  also  be 
selected  so  as  to  minimize  intercluster  traffic  which  is  prone  to 
utilize  longer  paths. 

The  above  considerations  can  easily  be  extended  to  an  m-level 
hierarchical  clustering.  Furthermore,  they  provide  us  with  guidelines 
as  to  the  choice  of  nearness  measures  and  clustering  techniques.  Before 
we  proceed,  let  us  notice  that  existing  nets  (e.g.,  ARPANET,  TRANSPAC) 
exhibit  the  above  features  to  a certain  extent.  In  particular,  nodes 
tend  to  be  localized  in  highly  populated  areas,  hence  they  constitute 
natural  groupings.  This  fact  is  more  likely  to  become  predominant  for 
large  networks,  and  this  is  a further  motivation  for  the  >flR  schemes. 
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5.1.2  Nearness  Measures 

In  order  to  cluster  the  set  of  network  nodes,  it  is  necessary 
to  have  some  numerical  similarity  neasurements  to  characterize  the 
relationships  among  the  nodes.  In  other  words,  a measure  of  association 
must  be  computed  for  every  pair  of  nodes  (or  for  subsets  of  nodes  in 
case  of  very  large  nets).  Any  such  measure  must  reflect  the  above 
properties  expected  of  a cluster,  with  the  understanding  that  strongly 
associated  nodes  are  more  likely  to  belong  to  the  same  cluster. 

The  above  properties  of  a good  clustering  may  be  simply  sum- 
marized by  stating  that  a cluster  must  preferably  be  composed  of  a 
subset  of  nodes,  as  highly  connected  as  possible,  and  which  interchange 
as  much  traffic  as  possible. 

The  above  considerations  lead  to  the  following  intuitive  near- 
ness measures.  Let  s„  denote  the  similarity  (nearness)  between  nodes 
i and  j.  The  most  obvious  similarity  measure  between  two  nodes  is  the 
inverse  of  the  hop  distance  between  them,  i.e.. 


(5.1) 


h^j  is  as  defined  in  Chapter  3. 

In  order  to  make  the  above  measure  sensitive  to  the  traffic 
pattern,  we  modify  Eq.  (5.1)  as 


s..  = lii 

^ h“. 

ij 


(5.2) 


Where  a is  a positive  scaling  exponent  which  reflects  the  importance  of 


the  variable  h. . as  compared  to  y. . in  the  evaluation  of  s. ..  a = 0 
il  ij  il 

means  that  s„  is  only  sensitive  to  the  traffic  rates. 


The  above  external  characteristic  of  a cluster  motivates  us  to 

seek  the  grouping  of  nodes  which,  seen  from  the  outside  (i.e.,  from  other 

nodes),  exhibit  some  sort  of  unity.  For  this  purpose,  we  define  a 

matching  variable  between  arbitrary  pairs  of  nodes  i,j  as  seen  from  an 

k 

arbitrary  node  k:  nu_.  . The  function  of  this  variable  is  to,  somehow, 

reflect  to  what  extent  the  routing  table  at  node  k indicates  the  same 

k 

route  (i.e.,  same  next-node)  for  traffic  destined  to  i or  j.  iik^  is, 

therefore,  closely  related  to  the  routing  scheme  utilized  and  can  be 

determined  either  in  a "static"  way  or  through  measurements. 

The  static  evaluation  of  iik  ^ relies  on  the  computation  of  the 

paths  which  result  from  a given  deterministic  routing  policy  (e.g., 

shortest  path  routing,  flow  deviation  method  [GERL  73A]  , etc.). 

k 

If  we  consider  a shortest  path  routing,  then  uk  ^ is  set  equal  to  one 

if  node  k indicates  the  same  next  node  on  the  paths  for  i and  j;  and  it 

is  set  equal  to  zero  otherwise.  Next,  we  define  the  nearness  between 

k 

nodes  i and  j as  the  average  of  nu  ^ over  all  nodes  k. 


s.  . 
ij 


m. 


N - 2 ij 

m 

Wj 


(5.3) 


As  mentioned  above,  mV^  can  be  determined  through  measurements 
such  as  the  ones  performed  on  the  ARPANET  [KLEI  74].  Such  measurments 
(of  interest  to  our  study  and  others)  have  been  carried  out  by  the 
Network  Measurement  Center,  NMC,  at  UCLA.  They  consist  of  collecting 
regular  snapshots  of  the  routing  tables  over  a certain  period  of  time. 

jc 

A matching  value  is  then  defined  for  each  snapshot,  say  y,  as  m.^(y) 
which  is  then  equal  to  one  if  node  k's  RT  shows  the  same  next  node  for 
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the  routes  to  i and  j (and  zero  otherwise) . 


of  snapshots,  then  is  defined  as 


If  Y is  the  total  number 


Finally,  s„  is  as  defined  in  Eq.  (5.3).  These  last  two  measures  will 
be  applied  below  to  the  ARPANET. 

Further  measures  may  be  defined  as  a combination  of  Eqs.  (5.2) 
and  (5.3),  or  else  as  based  on  other  network  characteristics.  In  this 
study  we  restrict  our  considerations  to  the  above  intuitive  and  simple 
measures  in  order  to  achieve  the  network  clustering. 


5.1.3  Clustering  Techniques 

The  nearness  measures  discussed  above  may  be  used  as  input  to 
some  clustering  techniques  which  then  realize  the  grouping  of  nodes  into 
clusters.  From  among  the  existing  clustering  techniques  [ANDE  73]  the 
Cooj)lete  Linkage  (CL)  method  seems  to  be  the  most  appropriate  to  our 
study.  This  method  belongs  to  the  family  of  hierarchical  clustering 
techniques.  It  operates  on  a similarity  matrix,  s * [s^],  to  construct 
a tree  depicting  specified  relationships  among  the  nodes.  More  pre- 
cisely, with  CL,  each  node  starts  as  a cluster.  Then,  at  each  step  the 
most  similar  pair  of  clusters,  say  p and  q,  are  merged  into  a new 
cluster,  called  t.  The  similarity  between  the  new  cluster  t and  some 
other  cluster  r is  determined  as  follows: 

str  “ min  {spr'  V*  (5'5) 
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As  such,  s is  the  similarity  between  the  two  most  dissimilar  pairs  in 
clusters  t and  r.  (Alternatively,  if  s„  is  a distance-like  measure 
(e.g.,  s.j  = ht then,  str  = max  {spr,sti>}). 

The  main  property  of  this  method  is  that  all  nodes  in  the  same 
clusters  are  linked  to  each  other  at  some  maximum  distance  or  minimum 
similarity.  As  a result,  this  method  tries  to  identify  "maximally 
connected  subgraphs"  which  is  precisely  our  main  objective. 

This  last  property  can  also  be  used  as  a stopping  criterion  by 
defining  a threshold  Th,  beyond  (or  below)  which  no  merging  is  allowed 
(i.e.,  if  si.  < Th  ¥ i,j  then  STOP). 

Another  stopping  criterion  may  consist  of  specifying  the  total 
number  of  clusters  (n^) . 

A shortcoming  of  CL  is  that  it  may  lead  to  various  non-optimal 
cluster  sizes  (see  Proposition  2.1)  which  may  considerably  reduce  the 
gains  in  table  length,  l,  obtained  from  optimal  size  clusters.  There- 
fore some  size  limitation  must  be  introduced.  This  can  be  realized  by 
not  allowing  the  merger  of  two  clusters  whose  combined  size  is  greater 
than  the  prespecified  limit.  This  limit  may  be  chosen  as  the  optimal 
integer  size  nj  (see  Proposition  2.6).  Furthermore,  CL  can  be  extended, 
as  shown  below,  to  accomodate  a hierarchical  clustering  with  more  than 
2 levels,  provided  we  are  given  a vector  of  maximum  degrees, 
n = (n^ , . . . , nm) . 

Similar  to  the  above,  let  p,q  be  the  most  similar  pair  of 
clusters,  which  are  such  that  p is  an  ith  level  cluster  and  q,  a jth 
level  cluster.  Also  let  n^(p)  (and  n^(q))  be  the  number  of  i-lSt  level 
cluster  (j-1  ) in  p (q) . Three  cases  to  consider  are. 
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(a)  i ■ j : Merge  p and  q into  t; 

If  m(p)  ♦ n^(q)  £ n^  , then  t is  an  ith  level  cluster  of 

degree  ni(t)  » n^p)  ♦ n^  (q) . 

St 

Else,  t is  an  i*l  level  cluster  of  degree  ni+1(t)  = 2. 

(b)  i < j : The  i**1  level  cluster  p is  augmented  with  j-i 

levels,  each  of  degree  one,  to  become  a j1*1  level  cluster; 
then  Step  (a)  is  performed. 

(c)  j < i:  The  same  as  above  for  q. 

s t 

Initially,  each  node,  say  p,  is  considered  to  be  a 1 level  cluster  of 

degree  nj(p)  ■ 1 (i.e.,  it  contains  one  0th  level  cluster).  Furthermore, 

St 

n^,  the  maximum  number  of  m-1  level  clusters,  is  usually  set  to  be 

s t 

large  enough  that  the  merging  of  two  m-1  level  clusters  always  results 
in  a m-lSt  level  cluster  (see  Step  (a)).  Either  that,  or  the  merging 
of  the  pair  p,q  is  not  allowed. 

The  above  clustering  technique  does  not  guarantee  that  cluster 
subnets  are  connected.  This  is,  however,  to  be  expected  of  such  a 
method  (see  below).  To  accomodate  for  this  deficiency,  a connectivity 
check  may  be  performed  before  allowing  the  merger  of  two  clusters. 

This  check  may  be  performed  by  using  a distance-like  measure  d^  which 
initially  is  set  equal  to  h^  and  after  a merging  of  clusters  p,q  into 
t,  then 


tr 


min  {d  , d } 

1 nr*  nrJ 


pr'  qrJ 


As  a result,  a merging  of  p and  q is  allowed  only  if  dpq=  1. 


Application  to  the  ARPANET 


The  above  modified  complete  linkage  method  (without  a connec- 
tivity check)  has  been  implemented  and  applied  to  the  ARPANET.  Fig.  5.1 
shows  a map  of  a 52  (duplex)  channel,  46  node  ARPANET  as  it  was  in 
operation  on  June  25,  1974.  CL  used  as  input  the  nearness  measures  of 
Eq.  (5.3)  as  computed,  first,  from  measurements  performed  at  UCLA  and 
second,  from  a shortest  (hop)  path  routing. 

(i)  Clustering  based  on  measurements.  Fig.  5.2  illustrates 
the  outcome  of  CL  (as  obtained  using  the  measured  nearness)  in  terms  of 
the  relative  table  length  A/N,  and  the  threshold  Th.  Th  is  defined  as 
the  minimum  nearness  between  any  pair  of  nodes  which  belong  to  a common 
cluster  at  any  level.  The  curves  are  shown  for  m = 2,  m * 3,  and  for 
a variable  m which  achieves  the  lower  envelope  for  fc/N.  For  m ^ 3,  the 

maximum  degrees  were  assumed  to  be:  n.  = n.  « ...  = n = 3,  as  moti- 

i i m-l 

vated  by  Proposition  2.5;  nm  was  left  unbounded. 

The  curve  corresponding  to  m = 2 was  obtained  with  no  con- 
straints on  (n^,  . It  reflects  the  fact  that  with  CL,  initially, 

l 1 

all  nodes  are  considered  to  be  clusters;  hence  Th=l,^=l+^x:l, 

and  at  the  last  step  of  CL  all  nodes  are  in  the  same  cluster.  Thus, 
l 1 

= 1 + ^ ~ 1 with  Th  « 0.11.  The  minimum  (with  m = 2)  of  i is  equal 
to  17  (recall  N = 46)  and  corresponds  to  the  clustering  structure 
shown  in  Fig.  5.3.  This  structure  is  composed  of  11  clusters,  with 
the  largest  cluster  containing  6 nodes.  The  suboptimum  l * 17  is  fairly 
close  to  the  theoretic  minimum  A *»  2^N^^1  = 14.  This  shows  the  fairly 
good  behavior  of  CL  in  this  particular  example.  Let  us  also  note  that 
at  that  minimum  point,  the  resulting  cluster  subnets  are  connected. 
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With  a variable  m,  the  global  suboptimal  point  is  reached  for 
£ s 14,  whereas  the  theoretic  global  minimum  is  £#  * 11. 

(ii)  Clustering  based  on  the  shortest  path  matching.  With  this 
measure,  CL  leads  to  the  following  suboptimal  points: 

m * 2 —>  £ =15 

m variable  — > £ =13 

They  represent  a slight  improvement  over  the  previous  ones.  For  the 
sake  of  comparison  with  the  earlier  outcome,  we  show  in  Fig.  5.4,  the 
clustering  structure  which  yields  £ = 17;  this  structure  is  also  composed 
of  11  clusters,  with  the  largest  containing  6 nodes.  The  clustering 
structures  of  Fig.  5.3  and  5.4  do  not  exhibit  strong  similarities.  This 
can  be  explained  by  the  quite  asymetrical  traffic  pattern  of  the  ARPANET 
[KLEI  74]  which  led  to  the  measured  nearness. 

The  evaluation  of  the  MR  schemes  based  on  such  clustering 
outcomes  is  the  object  of  the  next  section. 


5.2 


Simulation  of  the  MR  Schemes 


5.2.1  The  Simulator 

A PL1  simulation  program  has  been  developed1  to  help  investi- 
gate the  effects  of  hierarchical  adaptive  routing  on  the  performance  of 
arbitrary  networks.  The  computer  storage  and  conputational  requirements 
for  simulating  moderate  size  (a  few  hundred  nodes)  networks  placed  a 


*A  thorough  documentation  of  this  program  is  available  at  UCLA, 
Network  Measurement  Center. 


■ . -•* 
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restraint  upon  the  amount  of  detail  which  could  be  implemented  in  the 
simulator.  We  list  below  the  important  features  implemented  in  the 
simulator. 

1.  Messages  are  assumed  to  be  single  packet  messages  (hence, 
there  will  be  no  reassembly  problems)  whose  lengths  may  be  constant  or 
drawn  from  a truncated  exponential  distribution. 

2.  All  channels  are  assumed  to  be  duplex  channels  and  error 

free. 

3.  Each  node  (IMP)  may  be  composed  of  a finite  or  infinite 
number  of  S/F  buffers.  In  case  of  a finite  storage,  the  sharing  with 
maximum  queue  length  (SMXQ)  scheme  is  applied  (see  Appendix  B) . 

4.  The  routing  updates  are  periodic  and  occur  every  r seconds. 
The  updates  are  not  synchronized  among  nodes.  On  the  contrary,  initial 
update  times  are  selected  randomly  on  a node  pair  basis  (or  simply  on  a 
node  basis) . 

The  update  rule  is  performed  as  specified  in  Chapter  3 (for 
NCR  and  MIR  schemes)  except  that  in  Algorithm  3.9,  the  channel  length 
ats  is  now  a random  variable.  When  an  update  occurs,  the  channel 
length  ats  is  estimated,  as  is  presently  done  in  the  ARPANET.  If  we  let 
kj  be  the  line  capacity  index  and  k2  the  queue  index  (kj  * 4,  k^  * 1 
for  the  ARPANET),  then 

a^  = k.  + k_q^ 
ts  1 2Mts 

where  q is  the  current  queue  length  (in  packets)  for  line  (t,  s) . 

A further  modification  is  introduced  to  Algorithm  3.9  in  order 
to  avoid  chain  loops  [NAYL  75].  A change  in  routing  at  node  t is 
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allowed  only  if  entry  Q.(i)  at  s is  not  pointing  to  t.  Let  NN(s,  C.(i)) 
denote  the  next  node  field  at  entry  C (i)  of  node  s RT  (see  Fig.  3.1), 
then  the  first  step  of  Algorithm  3.9  is  modified  as 

IF  H(t,  C.(i))  > ata  ♦ HF(s,  C..(i))  AND  NN(s,  C..(i))  + t. 

5.  Acknowledgements  (ACK)  are  assumed  to  be  instantaneous.  A 
message  forwarded  from  one  node  to  another  can  be  rejected  if  there  is 
no  space  available  (recall  that  S/F  buffers  are  managed  by  the  SMXQ 
scheme) . This  situation  triggers  a negative  ACK  (a  departure  from  the 
ARPANET  protocol)  upon  reception  (instantaneous)  of  that  ACK,  the 
sending  node  reschedules  the  retransmission  of  that  message.  The 
message  is  deleted  when  a positive  ACK  is  received. 

6.  With  respect  to  the  flow  control,  messages  are  not  accepted 
in  the  net  for  either  of  the  following  reasons: 

(i)  There  is  no  buffer  space  at  the  originating  node. 

(ii)  The  total  number  of  messages  in  the  network  is  equal 
to  a given  critical  number,  NC. 

The  rejected  messages  are  considered  to  be  lost. 

This  terminates  the  description  of  the  main  features  of  the 
simulator;  we  now  proceed  with  some  applications. 

5.2.2  Simulation  of  the  ARPANET 

Our  previous  clustering  structures  of  the  ARPANET  (see  Figs. 
3.1,  3.3  and  3.4)  are  now  used  as  input  to  the  simulator  in  order  to 
conqpare  the  performance  of  the  NCR  and  the  MIR  schemes  (CER) . More 
specifically,  the  operating  conditions  are  as  follows: 
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Topology:  46  node  ARPANET,  Fig.  3.1.  It  is  composed  of  52  duplex 

channels  of  capacity  50  kbit/sec  per  channel,  except  for  lines  (16,  15) 
and  (45,  15)  which  are  of  capacity  230.4  kbit/sec.  The  propagation 
delay  on  the  lines  is  set  to  zero. 

mR:  The  net  is  operated  with  a non-clustered  (NCR)  adaptive  routing 

and  with  a 2- level  closest  entry  routing  (CER)  scheme.  The  underlying 
clustering  structures  of  CER  are  as  shown  in  Figs.  5.3  and  5.4.  In  what 
follows,  we  will  refer  to  the  representation  of  Fig.  5.3  as  AMC,  which 
stands  for  ARPANET  Measured  Clustering,  and  ASC,  ARPANET  Static  Cluster- 
ing, for  the  representation  in  Fig.  5.4.  The  non-clustered  version  will 
be  referred  to  as  ANC  (N  for  "non") . 

Traffic  Characteristics : We  assume  a uniform  traffic  matrix,  i.e., 

Yjj  » y V i,  j;  therefore,  the  network  throughput  is  equal  to 
N(N  - 1)y  * 2070  Y*  The  message  length  is  assumed  to  be  fixed  and 
equal  to  1 kbit. 

Update  Information:  kj  ■ 4,  kj  ■ 1.  Update  interval,  x = 0.64  sec. 

The  initial  update  times  are  chosen  randomly  on  a node-pair  basis.  The 
length  of  an  update  message  is  set  equal  to  1 kbit  with  or  without 
clustering. 

Node  Operating  Condition:  We  assume  an  infinite  nodal  storage,  and  a 

nodal  processing  time  equal  to  zero. 

The  results  of  the  simulation  are  shown  in  Fig.  5.5. 

Let  us  first  notice  that  the  maximum  throughput  is  achieved  for 
Ymax  * 0.274  msg/sec.  This  value  can  be  computed  by  considering  the 
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cut  set  (line  cut  set)  with  minimal  capacity  and  maximum  through 
traffic.  Channels  (24,  13),  (12,  47)  and  (48,  37)  partition  the  net 
into  two  equal  subsets  of  nodes  (23  nodes  each)  and  thus  they  constitute 
such  a cut  set.  The  traffic  in  one  direction  is  equal  to 


where  1/y  = l/yu  = 1 kbit  and  = 1/t  = 1.562.  The  capacity  of  this 
cut  (in  one  direction)  is  equal  to  150  kbits,  and  y = .274  msg/sec 


achieves  such  a maximum  traffic 


Fig.  5.5  shows  that  NCR  leads  to  a better  performance,  mainly 
for  Y _>  .2  msg/sec.  This  is  to  be  expected,  since  this  application  did 
not  account  for  the  savings  obtained  from  the  MR  schemes.  Such  savings 
are,  however,  relatively  small  due  to  the  small  size  of  the  ARPANET 


(46  nodes) 


Furthermore,  the  static  clustering,  ASC,  leads  to  a better 
performance  than  the  measured  clustering,  AMC.  This  is  mainly  due  to 
the  uniform  traffic  assumption. 

The  difference  in  performance  between  the  NCR  and  the  CER  may 
be  further  explained  as  due  to  the  low  connectivity  of  the  net,  which 
leads  to  chain  cluster  subnets.  The  average  distance  between  entry 
points  of  a cluster  is  approximately  3 for  an  average  network  path 
length  of  approximately  6. 

The  above  results  still  intuitively  indicate  that  for  an 
ARPANET  with  more  than  64  nodes  (at  which  point  the  RT  uses  more  than 
one  buffer),  one  may  expect  that  an  MR  with  a table  length  £~64  will 
start  to  show  some  ing>rovement  over  the  NCR  scheme. 


M/M/I 


Figure  5.5.  Simulation  of  the  ARPANET  as  Operated  with  an  MHR 


5.2.3 


mi 


Simulation  of  a 64-Node  Torus 

We  consider  in  this  section  a 64-node  torus  upon  which  we 
impose  two  clustering  structures  (see  Fig.  5.6).  In  this  particular 
example,  we  account  for  the  savings  obtained  from  the  reduction  in  table 
length;  furthermore,  we  use  the  overall  best  routing  (OBR)  scheme  to 
operate  the  network.  More  specifically,  the  operational  environment  for 
the  simulation  is  as  follows: 

Topology:  We  assume  a 64-node  torus,  composed  of  128  duplex  channels  of 

20  kbit/sec  each.  The  propagation  delay  on  the  lines  is  chosen  as  equal 
to  zero. 

MIR:  The  network  is  operated  with  an  NCR  and  an  OBR  scheme.  The  OBR 

scheme  is  utilized  with  two  clustering  structures:  a 2-level  clustering 

composed  of  16  clusters  containing  4 nodes  each  (i.e.,  ^ = 16,  nj  =4), 
and  a 3-level  clustering  with  degree  vector  n^  = n£  = n^  = 4.  These 
structures  are  shown  in  Fig.  5.6;  the  first  will  be  denoted  by  16  x 4C 
and  the  second  by  4 x 4 * 4C. 

Traffic  Characteristics:  We  assume  a uniform  traffic  pattern.  The 

offered  load  T is  then  equal  to  T = N(N  - l)y  = 4032y.  The  message 
length  is  governed  by  a truncated  exponential  distribution.  The  mean 
length  is  set  equal  to  500  bits  and  the  truncation  point  is  at  1000  bits. 

Update  Information:  Similar  to  the  above,  we  choose  kj  * 4,  * 1, 

x = .64  sec.  The  size  of  and  update  message  l/yu,  is  fixed  and  pro- 
portional to  the  table  length;  moreover,  we  assume  each  entry  to  be 
16  bit  long.  Therefore, 
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NC 

— > 

£ - 

64 

=> 

1/Vi  = 
u 

1024  bits 

16  x 4C 

— > 

£ = 

20 

“> 

1/p  * 

u 

320  bits 

4 x 4 x 4C 

— i > 

£ = 

12 

— > 

1/p  = 

u 

192  bits 

Node  Operating  Condition:  The  nodal  processing  is  set  to  zero.  Each 

node  is  assumed  to  contain  16  S/F  buffers.  These  buffers  are  shared 
between  the  4 outgoing  channels  under  the  SMXQ  sharing  scheme,  where  12 
is  the  maximum  length  of  any  queue. 

Results : Fig.  5.7  shows  the  results  of  the  simulation  under  the  above 

operating  conditions.  We  observe  that  OBR  with  the  "16  x 4"  2-level 
clustering  achieves  a better  performance  than  NCR  and  OBR  with 
4 x 4 x 4c.  The  difference  with  NCR  is  not  too  considerable;  this  is 
due  to  the  small  size  of  our  network  (64  nodes)  which  thus  does  not 
yield  to  great  savings  in  table  reduction.  However,  this  indicates 
that  clustering  starts  to  pay  off,  even  for  such  a small  network  size. 
Hie  OBR  scheme  with  the  4 * 4 * 4C  clustering  structure  achieves  the 
worst  performance;  furthermore,  is  saturates  faster  than  the  two  other 
cases.  This  is  due  to  the  fact  that  such  a degree  of  clustering  (high 
for  this  small  net)  leads  to  a significant  increase  in  path  length, 
hence  in  internal  traffic. 

Let  us  also  note  that  for  T ■ 1700,  the  network  gets  into  a 
deadlock  situation.  This,  and  the  bending  of  the  curves  at  T ■ 1600, 
is  due  to  the  buffer  limitation.  A better  flow  control  scheme  is 
required  to  operate  the  net;  recall  that  the  only  control  introduced 
here  is  in  terms  of  the  total  number  of  messages  which  can  be  in  the 
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network  at  any  given  time.  That  number  was  selected  to  be  equal  to 
819  (approximately  4/5  of  the  total  number  of  buffers  in  the  network) . 

5. 3 Conclusion 

In  this  chapter  we  mainly  laid  the  framework  and  introduced 
a methodology  in  defining  nearness  measures  between  pairs  of  nodes. 
Those  measures  were  utilized  with  a modified  conplete  linkage  method 
to  achieve  the  clustering  of  the  ARPANET.  Applications  to  larger  net- 
works should  be  investigated,  along  with  other  clustering  techniques. 

The  simulation  of  a 64-node  torus  confirmed  our  earlier 
theoretic  results  which  indicated  that  the  MiR  schemes  with  an  appro- 
priate clustering  structure  eventually  (as  the  network  grows  in  size) 
outperform  the  NCR  scheme.  The  transition  point  was  seen  here  to  be 
even  smaller,  in  the  neighborhood  of  64. 
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CHAPTER  6 


TOPOLOGY  DESIGN  CONSIDERATIONS  FOR  LARGE 
COMPUTER  COMMUNICATION  NETWORKS 

6.1  Introduction 

Up  to  now,  we  were  given  a large  distributed  computer  communi- 
cation network,  and  the  problem  was  to  devise  an  appropriate  adaptive 
routing  scheme  which  would  operate  efficiently  with  a fairly  small 
amount  of  routing  information.  Hie  study  of  the  MIR  routing  schemes 
showed  that,  indeed,  they  represent  a class  of  very  satisfactory 
solutions  to  the  routing  problem  in  large  networks.  Now,  going  one 
step  back,  the  problem  becomes  that  of  designing  the  topology  of  a 
minimum  cost  communication  network  which  connects  a large  set  of  nodes 
and  which  satisfies  a set  of  given  requirements  (traffic,  delay, 
reliability,  etc.).  Furthermore,  it  is  desirable  that  the  resulting 
network  be  fairly  easy  to  operate  (routing,  flow  control,  adaptation 
to  failures,  etc). 

The  bursty  nature  of  computer  traffic,  as  well  as  the  con- 
tinuously decreasing  cost  of  computer  hardware  [ROBE  74],  very  much 
favor  packet  switching,  as  currently  used  in  the  ARPANET  [ROBE  70], 
[HEAR  70] , [KLEI  70] , [FRAN  70] , [CARR  70] , [MGQU  74] , as  the  tech- 
nology for  us  to  consider.  Several  different  formulations  of  the 
topology  design  problem  can  be  found  in  the  literature.  Generally, 
they  correspond  to  heuristic  methods  with  different  choices  of 
performance  measures,  design  variables  and  constraints.  Along  those 
formulations,  several  solutions  have  been  proposed  [GERL  73A] 
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[FRAN  72  ] and  applied  to  ARPA-like  network  designs.  However,  for 
networks  with  sore  than  a few  hundred  nodes,  present  procedures  fail 
because  of  the  large  amount  of  computer  time  and  storage  needed  to 
perform  the  optimization.  Design  procedures,  based  on  hierarchical 
clustering,  have  been  proposed  [FRAN  73],  [GERL  73B],  [COVI  74]  to 
substantially  reduce  the  computational  cost  and  the  storage  require- 
ment involved  in  the  optimization  step.  Generally  speaking,  in  a 
two- level  hierarchical  design,  the  nodes  will  be  grouped  into  clusters 
and  "gates"  (special  cluster-exchange  nodes)  selected  from  each 
cluster.  Cluster  subnets  will  be  designed  separately,  and  then  a 
superaet  of  gates  will  be  designed  to  connect  the  clusters  together. 

The  assumption  is  that  nodes  in  the  same  cluster  are  most  likely  to 
be  very  few  hops  apart  in  either  a non-hierarchical  or  hierarchical 
design.  The  approach  described  briefly  above  could  be  easily  extended 
to  more  than  two  levels  in  the  hierarchy. 

In  this  chapter,  after  stating  the  design  problem  in  a more 
precise  form,  a class  of  hierarchical  design  procedures  will  be 
presented.  The  focus  will  be  upon  the  determination  of  a certain 
optimum  hierarchical  clustering  structure  of  the  set  of  the  nodes,  to 
be  used  in  the  design  phase.  In  other  words,  we  will  be  concerned 
with  the  choice  of  the  cluster  sizes  at  any  level  and  the  number  of 
levels  which  will  minimize  a certain  objective  function  (e.g.  computa- 
tional cost).  Some  related  questions,  such  as  the  decomposition  of  the 
global  performance  variables  and  requirements,  will  also  be  addressed. 

Frank  and  others  [FRAN  73]  showed  from  a feasibility  study  of 
a 1000  node  network  that,  indeed,  hierarchical  structures  are  desirable 


for  the  design  of  large  networks.  They  also  posed  the  same  questions 
concerning  the  clustering  structure  but  failed  to  answer  them  for  the 
general  case  (m  levels) . 

f 

6.2  The  Topology  Design  Problem 

The  same  network  model  as  in  Paragraph  6.1  is  used  in  this 

section. 


6.2.1 


Delay  Analysis 
Recall  Eq.(4.5) 


More  elaborate  expressions  for  T [FULT  72],  [KLEI  74]  may  be  obtained 
to  include  overhead  traffic,  nodal  processing  delay,  etc.  For  the 
purpose  of  illustrating  the  complexity  of  the  design  problem,  we  will 
limit  our  consideration  to  the  sinple  expression  above.  In  the  design 
problem,  T will  appear  either  as  the  objective  function  to  be  minimized 
or  as  a variable  constrained  to  be  less  than  or  equal  to  a given  value, 

^max‘ 

Let  fjk*^  be  the  average  flow  (bits/sec)  produced  in  channel  i 
by  messages  traveling  from  source  k to  destination  £ (flow  of  commodity 
(k,  A)),  and  let  f^  be  the  total  average  flow  in  channel  i,  given  by: 
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Consequently 


f.  .h. 

i P 


l NA  fi 

T ■ r £ cHtt 

1*1  1 1 


(6.1) 


Let  f * (fj,  £2*  • f^)  be  the  flow  vector,  and  let 
C * (Cj,  C2,  ...»  C^)  be  the  capacity  vector.  For  the  above  equation 
to  be  feasible,  the  vectors  f and  C must  satisfy 


f < C i.e.,  f.  < C. 
~ ~ 1—1 


(6.2) 


The  above  relation  will  serve  as  a constraint  in  the  design  problem. 


6.2.2  The  Comauiication  Cost 

To  the  i**1  link,  we  have  assigned  a channel  capacity  C^.  C 
is  a discrete  variable.  However,  for  the  development  of  effitient 
analytical  techniques,  it  is  often  convenient  to  approximate  C ^ with  a 
continuous  variable  [KLEI  64],  [GERL  73A]  during  all  (or  part)  of  the 
optimization  phase,  after  which  these  continuous  values  are  discretized. 
Typical  capacity  options  and  their  cost  are  listed  in  [GERL  74],  The 
cost  of  channel  i,  denoted  d^(C),  is  a function  of  the  capacity  and 
the  length  of  the  channel1.  In  general,  the  cost  shows  economies  of 
scale  with  respect  to  capacity,  and  it  increases  linearly  with  the 
length  of  the  channel. 


More  elaborate  charging  structures  such  as  ATT's  new  DDS  service 
have  also  been  devised. 


I 

l 
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6.2.3  Traffic  Requirement 

Average  "traffic  requirements"  between  nodes  can  be  represented 
by  a matrix  R 


IV 


Recall  that  Y ..  (messages/sec)  is  the  average  transmission  rate  from 
source  j to  destination  k.  In  general,  R is  given  as  an  input  param- 
eter to  the  problem.  In  some  cases,  R is  given  as 


R = pR 


where  R is  a known  traffic  pattern  and  p is  a variable  scaling  factor, 
usually  referred  to  as  the  traffic  level. 

As  a result  of  the  traffic  requirements,  the  following  multi- 
commodity  flow  constraint  [HU  69]  must  be  satisfied. 


N 

z 

j=l 


f(k,£). 

ji 


N 

z 


f(k,*) 

ij 


if  i = k 

if  i = £ 

otherwise 


(6.3) 


6.2.4  Reliability 

Links  and  nodes  in  a real  network  can  fail  with  non- zero  prob- 
ability, thus  interrupting  some  communication  paths.  It  is  important 
to  evaluate  the  overall  network  reliability  in  the  presence  of  such 
failures.  Several  reliability  measures  are  available:  the  probability 

of  the  network  being  disconnected,  the  fraction  of  communicating  node 
pairs,  the  connectivity  of  the  network,  etc.  [FRAN  72].  However,  no 
simple  satisfactory  measure  exists  yet.  In  the  design  problem,  for  the 
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sake  of  simplicity,  connectivity  is  usually  used  as  a reliability 


measure. 


This  concludes  our  model  description,  and  now  we  are  ready  to 
state  the  design  problem  more  formally  and  then  to  discuss  its  com- 
plexity. 


6.2.5 


The  topological  design  problem  can  be  defined  as  follows: 


Given:  Node  locations 

Requirement  matrix  R 
Cost-capacity  functions  d^(C) 


Minimize:  D(A,  C)  » £ d.(C.) 

ieA  1 


A,  C,  f 


(6.4) 


where  A is  the  set  of  arcs  which  corresponds  to  a specific  topology, 

C is  a vector  of  capacities  and  f is  a multicommodity  flow,  such  that 


(a)  f satisfies  the  requirement  matrix  R (se  (6.3)) 


.(b)  f < C 

1 ^i 

(c)  T - i Z 7~T-f-  1 

Y ieA  ci  *i 


(see  Eq.  (6.2)) 


(see  Eq.  (6.1)) 


(d)  Reliability  constraint:  e.g.,  K-connectivity. 


as*5  i 


Notice  that  the  above  cost  function  associates  the  entire  network  cost 
with  the  channels  themselves  [KLEI  64],  [GERL  73A].  This  does  not 
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represent  a loss  of  generality  with  respect  to  the  nodal  costs  which 
may  be  grouped  with  the  channel  costs. 

In  general,  there  are  possible  topologies.  Consid- 

ering all  possible  designs  by  a computer  is  out  of  the  question. 
Furthermore,  capacities  are  available  in  discrete  sizes.  This  means  t' 
an  enormous  integer  optimization  problem  must  be  solved. 

i* 

The  non-linearity  of  the  time-delay  functions  and,  in  some 
cases,  of  the  reliability  measure  add  another  dimension  of  complexity 
to  the  problem. 

There  exists  no  efficient  technique  for  the  exact  solution  of 

the  topological  design  problem.  Several  heuristic  procedures  have  been 

proposed  and  implemented.  Among  them,  we  mention  the  Branch  X-change 

method  [FRAN  72]  and  the  Concave  Branch  Elimination  method  [GERL  73B]. 

They  typically  start  with  an  initial  topology  over  which  they  perform 

some  alterations  in  the  course  of  the  optimization.  Built  into  those 

procedures  and  inherent  in  the  multicommodity  nature  of  the  flow,  is 

the  determination  of  the  shortest  path  between  any  pair  of  nodes  in  the 

3 

network.  This  operation  requires  approximately  up  to  the  order  of  N 

2 3 

operations  (N  to  N , N ■ number  of  nodes)  and  may  be  performed 
many  times  in  the  course  of  the  optimization.  The  overall  computational 

complexity  corresponding  to  those  heuristics  is  estimated  to  be  on  the 
order  of  N2 3 * *  to  N6  [FRAN  73],  [FRAN  72]. 

For  networks  with  more  than  a few  hundred  nodes,  present 
procedures  fail  because  of  the  large  amount  of  computer  time  and 

storage  needed  to  perform  the  suboptimization.  Furthermore,  for  such 


networks,  incremental  changes  in  the  number  of  nodes  ought  to  be 
considered  on  a local  basis  rather  than  on  a global  basis  as  it  is 
performed  now.  In  other  words,  addition  or  deletion  of  a node  should 
not  affect  the  topology  of  the  entire  network  to  a large  degree.  As  a 
result,  new  approaches  are  needed  to  deal  with  the  design  of  large 
networks . 

6.3  The  m-level  Hierarchical  Topology  (OTT)  Design  Procedure 

Various  tools  are  available  in  the  field  of  operations  research 
tp  solve  certain  classes  of  large  scale  mathematical  programming 
problems.  Among  those  techniques  are  the  Dantzig-Wolfe,  Bender's, 
Lagrangian  decomposition  methods  and  the  column  generation  method 
[LASD  70],  [HIMM  72],  [GEOF  74],  [GRAV  72].  Due  to  the  complexity  of 
our  design  problem,  the  direct  application  of  these  techniques  does 
not  seem  too  promising  and  even  less  practical.  Consequently,  a good 
heuristic  method,  based  on  intuition  and  on  the  natural  properties  of 
the  problem,  is  desirable.  However,  exact  design  procedures  should  be 
developed  in  order  to  serve  as  comparison  for  the  evaluation  of  any 
heuristic  design  technique.  Hierarchical  design  procedures  are  good 
candidates  to  alleviate  the  tremendous  complexity  of  the  problem  and 
yet  produce  some  good  topological  design.  Such  a procedure,  the 
m-level  hierarchical  topology  (WT)  design  method,  will  be  presented 
and  studied  in  the  rest  of  the  chapter. 

The  idea  behind  the  htiT  is  to  impose  a decomposable  structure 
on  the  design  problem  which  will  result  in  a set  of  smaller  sub- 
problems. In  other  words,  we  will  introduce  independencies  among 


subsets  of  design  variables.  The  imposed  independencies  will  substan- 
tially reduce  the  set  of  feasible  solutions  and  also,  as  a direct 
consequence,  the  computational  cost.  In  doing  so,  there  is  the  risk 
of  eliminating  the  optimal  solution.  Therefore,  it  is  very  important 
to  seek  "natural"  decompositions. 

The  four  major  steps  in  an  MHT  are: 

a.  m- level  hierarchical  clustering  (MIC)  of  the  set  of  nodes. 

b.  Decomposition  of  the  objective  functions  and  the  design 
constraints. 

c.  Decomposition  of  the  optimization  problem  into  a set  of 
smaller  subproblems.  Solve  for  the  set  of  subproblems 
with  possible  iterations. 

d.  Exact  performance  evaluation  of  the  resulting  network 

and  comparison  with  optimum  performance. 

6.3.1  Step  a:  MHC  of  the  Set  of  Nodes 

The  decomposition  mentioned  above  will  be  realized  through  an 
m level  hierarchical  clustering  of  the  set  of  nodes,  based  on  some 
appropriate  nearness  measure.  As  defined  in  Chapter  2 (see  Fig  2. 
and  Paragraph  2.3.1  for  definitions  and  notation),  the  MIC  consists  of 
grouping  the  network  nodes  (0^  level  clusters)  into  1st  level  clusters 
which  in  turn  are  grouped  into  2°^  level  clusters.  This  operation 
continues  in  a bottom  up  fashion  until  the  grouping  of  the  m-2n<*  level 
clusters  into  m-lst  level  clusters  whose  union  constitute  the  m**1  level 
cluster.  The  m**1  level  cluster  is  the  highest  level  cluster,  and  as 

such,  it  includes  all  the  nodes  of  the  network.  This  partitioning, 
easily  describable  by  a tree  structure  as  in  Fig.  2.4,  could  also 
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proceed  in  a top  down  manner. 

The  above  structural  similarities,  with  the  NHC  devised  for 
the  adaptive  routing  in  large  networks,  do  not  quite  carry  over  to  the 
choice  of  the  nearness  measures  and  probably  not  to  the  clustering 
techniques,  both  of  which  will  greatly  affect  the  outcome  of  the  WT. 
The  nearness  measures  must  take  into  account  the  cost  of  the  different 
components  of  a communication  network  (switching  nodes,  channels, 
etc.),  the  traffic  and  reliability  requirements,  the  delay  of  a message 
in  the  net,  etc.  Some  examples  of  simple  nearness  measures  are: 

(i)  s = •ji-  nearness  between  nodes  i and  j 

3 ij 


(ii) 


s.  . 
ij 


Yil 


where  d^  is  the  geographical  distance  between  nodes  i and  j; 

Y-^  is  the  rate  of  traffic  from  i to  j,  and  the  exponent  e is 

between  zero  and  one. 

Here  also,  the  direct  application  of  the  clustering  techniques 
[ANDE  73],  [DUDA  73]  may  lead  to  various  non-optimal  cluster  sizes 
which  will,  in  general,  considerably  reduce  the  computational  gains 
[KERN  70]  obtained  frc”  optimal  size  clusters.  One  way  to  decide  upon 
an  MIC  structure  is  to  choose  the  one  which  will  minimize  the  computa- 
tional cost  incurred  in  the  WT.  This  problem  will  be  posed  and  solved 
later  in  this  chapter. 

Along  with  the  me,  we  must  select  the  gates  (exchange  nodes) 
for  all  clusters  at  all  levels  (Fig.  6.1).  The  function  of  the  gates 
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from  a given  cluster  is  to  handle  the  traffic  exchanged  between  the  set 
of  nodes  in  that  cluster  and  those  outside.  More  specifically,  the 
assumption  underlying  the  flow  of  messages  is  as  follows. 

Flow  Assumption  6.1 

a.  Traffic  between  nodes  in  the  same  cluster,  at  any  level, 
will  only  take  paths  which  are  internal  to  that  cluster,  i.e.,  paths 
contained  in  the  corresponding  local  subnetwork. 

b.  Traffic  between  nodes  in  different  k**1  level  clusters 

§t 

(k  = 1,  ...»  m - 1),  but  which  belong  to  the  same  k+1  level  cluster, 

s t 

will  first  be  channeled  to  a k+1  level  gate  of  the  originating  clus- 

s t 

ter  over  its  local  subnetwork;  then,  it  will  take  the  k+1  layer 

St 

subnetwork  of  gates  to  reach  a k+1  level  gate  of  the  destination 
cluster,  at  which  point  it  will  be  dispatched  over  the  local  subnetwork 
to  finally  reach  the  destination  node.  (This  is  the  standard  procedure 
in  hierarchical  networks.) 

Fig.  6.1  illustrates  some  of  the  preceding  and  forthcoming 
definitions  for  a 3-level  hierarchical  design. 

A local  subnetwork  of  a k**1  level  cluster,  say  C^,  is  defined 
recursively  as  the  collection  of  those  local  subnetworks  associated 
with  each  k-lst  level  cluster  {(^  j (i) } ^ which  belongs  to  and  which 
are  connected  together  through  a subnetwork  of  k**1  level  gates  called 
a k**1  layer  subnetwork.  Consequently,  such  a local  subnetwork  is 
composed  of  k hierarchical  layers,  each  of  which  is  composed  of  a 
certain  number  of  layer  subnetworks.  As  a reminder  of  the  k layers, 
the  subnetwork  will  also  eccasionally  be  called  a k- level  subnet  or  a 
k- level  local  subnet. 
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The  selection  of  gates  at  all  levels  is  governed  by  the  fol- 


lowing assumption. 


Gate  Assumption  6.2 

Given  an  integer  vector  B = (6,,  B~,  ....  B ) where  B,  = 1 

— x i m i 

and  a selection  rule,  then,  starting  at  k = 1,  select  Bk+jk+lst  level 
gates  among  the  set  of  kth  level  gates  of  each  kth  level  cluster 
C^fi^,  ...»  Repeat  this  step  sequentially  until  k = m - 1.  A 

network  node  is  considered  to  be  a 1st  level  gate. 

Let  ek  be  a kth  level  gate,  and  Ek(im,  i^,  ....  ik+J)  * 

{®k  / e Sc^m*  *jn-l*  * " ’ ^k^  ^e  t*le  set  ^eve^  gates  in 

the  corresponding  cluster.  Also,  let  g.(i  , i ,,...,  i.  ,)  be  the 

K m IB-1  K*1 

size  of  the  abovementioned  set  of  gates.  As  a consequence  of  the  gate 
assumption,  the  following  relation  exists  between  the  size  and  the 
degree  vectors. 


Mv 


1m-l* 


1k+l) 


nk(im' 


*k+r 


(6.5) 


Notice  that  for  k = 1,  the  set  of  1st  level  gates  of  a 1st  level 
cluster  is  simply  the  set  of  all  the  network  nodes  contained  in  that 
cluster,  hence 


«j<V 


m-l’ 


‘a’  • 


Vv 


m-l' 


V 


which  explains  the  convention,  Bj  = 1.  For  the  above  selection  scheme 
to  be  feasible,  the  number  of  k*^1  level  gates  of  a k^1  level  cluster, 

ji  i • c • > 


must  be  no  less  than 


*k+ 
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W x«-i»  ^i3  - 


Vk  = 1,  . . . , m - 1 


The  above  relation,  combined  with  Eq.  (6.5),  will  henceforth  introduce 
a constraint,  to  be  referred  to  as  "gate  constraint,"  over  the  choice 
of  the  degree  vector  n characterizing  the  KHC  structure. 


Gate  Constraint 


nk(im-  Vl- 


...  ik+1)  ¥k  • 1,  2,  ....  m - 1 

¥ (V  Vl'  Xk+13  f6,6) 


The  choice  of  the  vector  8 will  be  mainly  related  to  the 
reliability  constraint,  Eq.  (6.4).  If  a K-connectivity  is  to  be  im- 
posed on  the  topology  of  the  network,  then  the  vector  8 must  be  such 
that  8^  K for  i = 2,  3,  ...,  m.  This  is  obvious  since  the  set  of  the 
8^  it31  level  gates  of  an  i-lSt  level  cluster  represents  a cut  set 
[HARA  72]  for  the  other  nodes  in  that  cluster. 

Let  a centralized  hierarchical  network  be  defined  as  a hier- 
archical network  whose  topology  is  such  that  all  kth  level  clusters 
(for  all  k's)  communicate  with  the  outside  through  only  one  k+lst  level 
gate.  Consequently,  the  corresponding  vector  8 is 

8 = (1,  1,  ...,•  1)  i.e. , 8i  = 1 ¥ i = 1,  ...»  m 

This  structure  is  commonly  found  in  the  hierarchical  star  networks  and 
also  in  the  organization  of  many  business  or  government  agencies. 
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6.3.2  Decomposition  Step 

The  main  computational  gains  introduced  by  the  WT  come  from 
the  separate  design  of  the  different  components  of  the  networks,  i.e., 
the  design  of  the  layer  subnets  at  all  levels.  To  perform  this  opera- 
tion, it  is  necessary  to  define  performance  measures  for  those  subnets 
as  well  as  the  corresponding  constraints  to  be  inposed  on  their  design. 
Then,  the  global  objective  function  and  constraints  must  be  expressed 
in  terms  of  those  associated  with  the  layer  subnets.  As  an  exaaple, 
assuming  that  messages  travelling  over  kth  layer  subnets  will  incur 
the  same  delay,  T^,  then  the  problem  is  to  express  the  global  average 
delay,  T,  in  terms  of  T^'s,  i.e.,  an  expression  of  the  form 


m 

£ 

i*l 


aiTi 


must  be  derived.  A particular  case  is  solved  in  Section  6.7  when 
an  optimal  NWC  structure  is  considered.  Similarly,  given  a certain 
dollar  budget,  the  question  may  arise  as  to  how  much  one  should  allo- 
cate for  the  design  of  each  of  the  layer  subnets  at  any  level.  Section 
6.7  will  address  some  of  the  above  questions. 

6.3.3  Design  and  Evaluation  Steps 

These  two  steps  will  not  be  studied  in  depth  in  this  report. 
However,  the  economies  of  hierarchical  networks,  as  well  as  their 
operational  flexibility  (routing,  flow  control,  addition  of  nodes) , 
will  be  pointed  out  in  Chapter  7.  Since  the  MHT  results  in  the  design 
of  a set  of  much  smaller  subproblems,  then  present  heuristic  design 
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methods  [GERL  73A] , [FRAN  72]  become  feasible  and  quite  attractive  to 
perform  the  required  job. 

In  order  to  evaluate  the  computational  conplexity  in  the 
design  phase  of  the  NWT,  we  will  make  the  following  assumption: 
Computational  Cost  Assumption  6.3 

a.  The  computational  cost  incurred  in  the  design  of  a k**1 

th  “v 

layer  subnet  connecting  a set  of  n k level  gates  is  equal  to  n K 

(k  * 1,  2 , • . • i m) . 

b.  The  total  computational  cost  involved  in  the  design  is 
equal  to  the  sum  of  the  costs  induced  in  the  design  of  all  the  layer 
subnets . 

Recall  that  the  polynomial  form  of  the  computational  cost  is 
the  one  normally  used  [FRAN  72]  to  characterize  the  computational 
complexity  of  most  of  the  present  design  algorithms.  The  fact  that 
different  exponents,  a^'s,  could  be  selected,  depending  on  the  level 
of  the  hierarchy,  is  provided  to  allow  the  modelling  of  the  design  of 
hierarchical  networks  where  different  technologies  or  design  algorithms 
or  both  are  considered  at  each  level  or  group  of  levels. 

In  summary,  the  main  elements  involved  in  the  computational 
co^>lexity  of  the  HIT  have  been  exposed  under  a set  of  assumptions 
which,  hopefully,  retain  the  essential  character  of  the  class  of  hier- 
archical design  procedures. 

6.4  Optimal  Clustering  Structure,  General  Case 

Given  the  number  of  levels,  m,  the  objective  here  is  to  deter- 
mine the  optimal  clustering  structure,  i.e.,  the  degree  vector  n. 


which  will  result  in  a minimum  conputational  cost.  A similar  problem 
has  been  solved  in  Chapter  2 but  with  a different  objective  function, 
i.e.,  the  length  of  the  routing  table  (see  Eq.  2.3)). 


6.4.1  Expression  of  the  Computational  Cost  and  Problem  Statement 

The  model  aimed  at  capturing  the  computational  complexity  of 


the  NflT  has  been  introduced  and  discussed  in  the  previous  paragraph. 

Let  G(m,  n,  a,  8)  represent  the  computational  cost  incurred  in 
the  design  step  of  the  MHT.  It  is  a function  of  the  underlying  clus- 
tering structure,  the  number  of  gates  and  the  conputational  cost  of 
the  design  of  the  different  layer  subnets  which  are  respectively  char- 
acterized by  (m,  n),  8 and  a.  Some  of  these  variables  will  be  dropped 
from  the  above  notation  when  no  confusion  is  possible.  Notation  such  as 
G,  G(m,  n),  etc.,  will  be  encountered  in  the  rest  of  the  chapter. 
Similarly,  G^fm,  n,  a,  8)  will  denote  the  cost  incurred  in  the  design 
of  all  k**1  layer  subnets.  We  are  now  ready  to  evaluate  G;  we  start 
with  the  simple  case  of  a 3-level  clustering. 


6.4. 1.1  Expression  of  G for  a 3-Level  Hierarchy 

Fig.  6.1  shows  a 3-level  hierarchical  clustering  of  a set  of 
nodes.  From  Assumptions  6.2  and  6.3  and  Eq.  (6.5),  the  computational 
cost  incurred  in  the  design  of  all  the  Is ^ layer  subnets  is 


n3  n2(*3J  a. 

G1(3,  n,  a,  8)  = t6ini(i3»  i^J  1 
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where  the  summations  extend  over  all  1st  level  clusters  of  the  network. 
(Recall  that  Bj  = 1).  Similarly 

n3  a, 

g2(3»  n.  o,  B)  = Z [82n2(i3)] 


V1 


and 


g3(3,  n,  a,  B)  = [B3n3] 


a. 


Consequently,  the  global  computational  cost  is 

n3  n2(i35 

G(3,  n,  a,  6)  = Z £ [Bjn^ij,  i2)  ] 1 


a. 


x3=1  x2=1 


n. 


3 a2  a3 

+ X,  IWW  * t03n3] 

13"1 


(6.7) 


6.4. 1.2  Expression  of  G for  an  m-Level  Hierarchy 

From  Assumption  6.3  and  Eq.  (6.5),  the  computational  cost  of 
the  design  of  a layer  subnet  is  equal  to 

tgk(V  1m-l*  •”»  xk+l^  = ^knk(lm’  1m-l*  •*'*  ik*l^°k 

Hie  above  expression,  summed  over  all  the  k^1  level  clusters,  repre- 
sents the  computational  cost  incurred  in  the  design  of  all  the  k1*1 
layer  subnets,  i.e., 

nm  nm-l(V  nk+l(lm*  Vl»  *•*»  ik+2)  ol 
Vm-  n.  a.  6)  - £ £ •••  £ [Bk»k(V  ‘.-l Vl» 

m m-1  k+1 


G (m,  n,  a,  B)  = (Bn) 

iis  ***  **»  III  III 


(6.8) 
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Notice  that  for  k = m,  by  convention,  we  will  set  the  multiple  summation 


to  one. 


From  Assumption  6.3,  the  global  computational  cost  is: 


m 


G(m,  n,  a,  6)  = 21  G,  (m,  n»  a»  3) 
~ ~ ~ k=l  * ~ ~ ~ 


(6.9) 


6.4. 1.3  Problem  Statement 


given:  N,  a,  $ 

minimize:  G(m,  n,  a,  6)  [see  Eq.  (6.9)] 

over:  m and  n 

^subject  to:  size  constraint  [see  Eq.  (2.1)] 

gate  constraint  [see  Eq.  (6.6)] 
m positive  integer  variable 
n vector  of  positive  integer  variables 


(6.10) 


The  precise  problem  expressed  above  is  a non-linear  integer  programing 
problem  which  has  not  been  solved  in  its  entirety.  In  order  to  make 
progress  toward  a solution,  we  choose  to  relax  some  of  the  constraints, 
namely,  the  integer  and  gate  constraints.  It  is  also  necessary  to  tem- 
porarily freeze  the  variable  m.  The  relaxation  of  the  gate  constraint 
is  not  of  too  much  consequence,  since  it  will  be  shown  that  for  prac- 
tical values  of  6,  a,  N,  m,  the  optimal  solution  will  satisfy  that 
constraint.  The  added  relaxation  of  the  integer  constraint  will  lead 
to  a nice  analytic  solution  when  m is  given;  (for  some  particular 

properties  of  the  vectors,  a and  0,  an  analytical  solution  is  found  for 

«*» 

the  optimal  m. ) The  resulting  real-valued  solution  is  of  considerable 
inqportance  for  at  least  two  reasons:  (1)  The  study  of  its  behavior. 
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with  respect  to  the  variables  m,  a,  3,  will  provide  us  with  insight  as 
to  how  much  computational  gains  can  be  obtained  through  the  application 
of  the  NUT  design  procedure  and  into  the  choice  of  the  appropriate 
clustering  structure.  (2)  A sub-optimal  integer  solution  could  be 
directly  obtained  from  the  real -valued  solution  (thereby  providing  an 
upper  bound  on  the  cost) . As  a consequence  of  the  relaxation  of  the 
integer  constraint,  one  question  arises  as  to  what  is  the  meaning  of  a 
discrete  summation  where  the  upper  variable  is  not  an  integer.  Con- 
sider as  an  example 


The  summation  is  only  meaningful  in  two  instances 


(1)  when  n,  is  integer 


The  solution  of  the  optimization  problem  will  show  that  clusters  at  the 
same  level  must  be  of  equal  degree  (size);  hence,  the  2nd  condition  will 
always  be  satisfied  a posteriori. 

The  optimization  problem  to  be  solved  is  reduced  to: 


minimize 


over:  n 


s.t.:  size  constraint 

n ^ 0 real -valued  vector 

m > 0 integer 


6.4.2 


)timal  Solution 


Proposition  6.1 

Given  m,  the  number  of  levels  in  the  hierarchy,  and  assuming 
that  cl  > 1 for  all  i = 1,  ...,  m , then  the  solution  of  the  optimiza- 
tion problem  6.11,  i.e.,  the  optimal  clustering  structure,  is  such  that: 
(a)  All  clusters  at  the  same  level,  k = 1,  ....  m,  are 
composed  of  an  equal  number  of  lower  level  clusters;  i.e.,  all  nodes 
at  the  same  level  in  the  tree  representation  are  of  equal  degree. 

The  optimal  degree  vector  reduces  to  an  m-dimensional  vector, 
n = (n^.n^,  ...»  n^)  whose  components  are  the  solution  of  the  following 
set  of  difference  equations: 

nk(im*  Vl*  VP  = nk  * k - 1.  ■ and  (i  , ...,  i ) 


1 ™ 

A • • • n 

2 3 m 


n a.  __k_ 
i=l  ID,., 


■ n (a.  - 1)  i=l  1 1 


(6.12) 


k = 2,  3,  ...,  m 


where,  by  convention  Jl  n4  = 1 J and  D,  is  the  solution  of: 


i=m+l 


°2  3 1 


(6.13) 


Dk  3 VlVl  * n («i  - 1)  k > 3 
1=1 


Also,  is  the  solution  of 
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a. 


B2  = N 


rBkiDk  A-A-i 

tod  Vi 


k-2 

k-2  - n (a.-i) 

n (ol  - Di j-  ’ 

i=l 


(6.14) 


Lc^k-i)0^-1  -l 


- i=l  * 

(•— 1 

k-i) 

k >_  3 

(b)  With  this  optimum  solution,  the  minimum  computational 


cost  is: 


G(m,  a,  3)  = B 


m+1 


(6.15) 


The  proof  of  Proposition  6.1  is  given  in  Appendix  C.  It 
consists  of  showing  when  all  other  variables  are  fixed,  that  the 
nj(*)'s  must  all  be  equal  (to  n^).  Then,  n^  is  replaced  by  its  optimal 
expression  in  the  objective  function,  and  the  same  operation  is  repeated 
for  the  degrees  at  the  next  level,  and  so  forth,  until  all  levels  are 
exhausted. 

As  a consequence  of  Proposition  6.1,  the  optimal  size  vector 
also  reduces  to  an  m-dimensional  vector,  g = (g^,  g^,  ...,  g^) , whose 
components  are  given  by 


• 6k\ 


k = 1,  , m 


(6.16) 


where  n^  is  the  solution  of  Eq.  (6.12). 


The  computation  of  the  variables  D^,  B^,  n^  may  proceed 
sequentially  by  computing  the  sequences  below. 


(1) 

°3’ 

^4 » • • • » 

D . 
m+1 

B3 

B4 

B . 
m+1 

°3* 

D/ 

4 

D , 
m+1 

(3) 

n , 
m 

n i i • • 
m-l* 

. , n^ 

where 


m 


n = 
m 


_m-l  D 

n c«i  - i)  g 

i=l  m 


a 


a (3  ) 
nr  nr 


m 


D J 

m 


m+1 


(6.17) 


nm  is  the  first  element  to  compute  in  the  sequence  {n^} 

Notice  that  one  can  also  solve  the  difference  equations, 
(6.13)  and  (6.14)  to  obtain  explicit  expressions  for  Dj,  and  B^/D^. 

Explicit  Expression  of 


* A"1  \ / 3-2  > 

Dk  = L ( n Qti ) ( IT  (a.  - 1) 
k j-2\i-j  / \ i=l  1 J 


¥ k > 1 


(6.18) 


By  convention,  when  the  lower  index  in  the  product  exceeds  the  upper 
index,  then  the  product  is  set  to  be  equal  to  one.  If  the  same  situa- 
tion arises  with  a summation  sign,  then  the  sum  is  set  to  be  equal  to 
zero.  This  convention  will  apply  throughout  the  chapter. 


Explicit  Expression  for  B^/D^ 


rn  -A.  k-1 

ft]  -n 

1=2 


a. 

3 


-a.D. 
J J 


r J-1 

n («i  - i) 

i=l 


a. 


(3J 


3-1 

n (a.-l) 
i=l 


k-1 

n a. 

i= j+11 


(N) 


k-1 

.n.ai 

i=l 


¥ k > 2 (6.19) 
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The  proof  follows  immediately  from  Eq.  (6.14). 

The  combination  of  Eqs.  (6.18)  and  (6.19),  at  k = m + 1, 
will  provide  an  explicit  expression  for  the  minimum  computational  cost, 
G(m,  a,  8).  The  next  natural  step  to  take  is  to  perform  the  optimiza- 
tion of  G with  respect  to  m.  This  may  easily  be  done  numerically  when 
m is  restricted  to  a certain  range  of  integer  values. 

Numerical  Examples 

Define  G = (Gj,  G2>  Gy  G^) . Recall  that  G^  is  the  computa- 
tional cost  for  the  design  of  the  kth  layer  subnets. 

a = (2,  2.5,  3,  3.5) 

(i)  6 = (1,  1,  1,  1) 

n * (103)  i.e.,  N = 103 

6 A ai 

G = 10°  (G  = N for  m = 1) 

n = (25.08  , 39.86  ) Recall  that  n 

is  assumed  to 
be  continuous. 

G = (2.51  104,  1.00  104) 

G = 3.51  104 


m = 1 


m = 2 


m «=  3 n = (8.65  , 9.63  , 12.00  ) 

G * (8.646  103,  3.453  103,  1.728  103) 

G * 1.38  104 

m 3 4 n » (5.46  , 5.22  , 5.58  , 6.29  ) 

g - (54. 6T  102,  21.85  102,  10.92  102,  6.24  102) 
=»  9.36  103 


i 


G 


i 


[ 

* 


? 


(ii)  g = (1,  2,  2,  2) 

m = 1 n (103) 

G = 106 

m = 2 n * (41.16  , 24.30  ) 

jg  - (41.16  , 48.60  ) 

G = (4.117  104,  1.647  104) 

G = 5.76  104 

m = 3 n * (17.29,  7.65 
g * (17.29,  15.30 

G * (1.73  104,  .69 
G = 2.77  104 

m = 4 n = (12.26  , 4.83  , 4.26  , 3.96  ) 

g ■ (12.26  , 9.66  , 8.52  , 7.92  ) 

G * (1.22  104,  .49  104,  .25  104,  .14  104) 

G * 2.10  104 

6,5  Optimal  Clustering  Structure  with  Uniform  Design  Strategy  and 

Gate  Assignment 

This  section  deals  with  the  special  case  where  the  same  design 
procedure  is  used  at  all  levels  of  the  hierarchy.  Also,  an  equal  nunfcer 
of  gates  is  selected  from  clusters  at  all  levels,  i.e.. 


, 7.56  ) 

, 15.12  ) 

4 4 

10  , .34  10  ) 
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~i  i»|iiimiiii^|inf i rji  i m w^iii 


a.=a  Vk  > 1,  a>l 

* ” (6.20) 

8k  = 6 ¥k>  2,  8j  A 1 

As  a result,  much  simpler  expressions  are  obtained  for  n and  G at 
optimality,  given  m.  Furthermore,  it  is  possible,  analytically,  to 
find  the  optimum  number  of  levels  in  the  hierarchy.  At  global  opti- 
mality, all  the  layer  subnets  must  be  of  equal  size.  This  surprising 
result  has  an  intuitive  explanation  given  below. 

In  what  follows,  first,  the  expressions  for  n and  G will  be 
derived  at  optimality,  given  m.  Then,  the  global  optimum  solution 
will  be  determined  and  its  peculiar  properties  will  be  studied,  as  well 
as  its  behavior  with  respect  to  a and  8.  Finally,  comparisons  with 
intuitive  feasible  solutions  will  be  made. 


, 


6.5.1  Optimal  Expressions  Given  m 
Corollary  6. 1 

Given  m,  the  number  of  levels  in  the  hierarchy,  and  assuming1 
that  ou  = a for  i * 1,  2,  . . . , m;  a > 1;  and  8^  = 8 for 

i * 2,  3,  ...,  m ; then,  the  optimal  solution  of  Problem  6.11  is 


244 


With  this  solution,  the  minimum  computational  cost  is: 


G(m,  a,  $)  B D 


m+1 


°‘(C‘-1)DB/(a  - 


(•a)  (■ 


a” 


a 


m-1 

) 


m+1 


(6.22) 


where 


D.  = ak_1  - (a  - l)k_1  for  k > 1 


(6.23) 


Proof: 


To  prove  Corollary  6.1,  we  need  to  find  the  expression  for 

\ and  VDk- 

Expression  for 


From  Eqs.  (6.18)  and  (6.20) 


Dk  = E ak"j(a  - ljj'2 
j=2 


(6.24) 


k-2 


No.  ^ - (a  - l)k‘2  £ (yfy) 

After  summing  on  i,  we  find  Eq.  (6.23). 

Expression  for  B^/D^ 

From  Eq.s  (6.19)  and  (6.20) 


r“k  1 °k 
V -* 


-Eiok-i-2(c.-.)i  E^-Vo-D1-' 

(a  - l)i-1  e1"2  Na 


k-1 


To  evaluate  the  exponents  involved  in  the  above  expression, 
we  need  the  following  identity. 


1 


The  same  result  as  in  Eq.  (6.21)  has  been  derived  in  [FRAN  72]  for  m * 2 
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Finally,  the  exponent  of  3 is  equal  to  a(a  - l)[^  After  substi- 
tuting these  exponents  into  the  above  expression  of  B^/D^  and  grouping 
terms  together,  we  get: 


From  the  above  equation  at  k = m + 1 and  Eq.(6.15)  we  obtain 


The  proof  of  Eq.  (6.21)  will  proceed  by  induction.  First 
let  us  observe  that  from  Eqs.  (6.12),  (6.20)  and  (6.26) 


Factoring  out  the  terms  in  a,  a - 1,  8 and  using  Eq.  (6.23)  to  evaluate 
their  exponents  (to  be  denoted  by  exp  ( )),  we  find 


After  substitution 


L J L J (6.27) 

For  k « m,  the  product  term  in  Eq.  (6.27)  vanishes  to  one,  and  the 
resulting  expression  of  n^  satisfies  Eq.  (6.21).  Assuming  that  Eq. 
(6.21)  is  true  for  i = m,  m - 1,  ....  k ♦ 1.  let  us  show  that  it  i* 


From  the  induction  hypothesis  and  Eq.  (6.24) 


Substituting  Eq.  (6.28)  into  Eq.  (6.27),  and  using  Eq.  (6.23)  to 
sinplify  the  resulting  exponents,  we  arrive  at  the  expression  of  nfc 
given  in  Eq.  (6.21).  Since  Eq.  (6.27)  is  only  true  for  k * 2,  3,  . 
then  it  remains  to  be  proven  that  nx  satisfies  Eq.  (6.21).  This  is 
obvious  from  the  expression  of  n.  in  Eo.  C6.121  and  f-mm  f n r a aai 


in  «■ 


Remarks 

(i)  Size  vector:  From  Eqs.  (6.16)  and  (6.20) 

gl  * nl»  *k  “ Pnk  k ■ 2,  3,  ....  ra 

Substituting  Eq.  (6.21)  into  the  above  expression,  we  get  a unique 
expression  for  gR,  valid  for  k = 1,  2,  ...,  m. 

(ii)  Gate  Constraint:  To  satisfy  the  gate  constraint, 

Eq.  (6.6),  the  vector  n must  be  such  that 

nx  _>  8,  n^  _>  1 k « 2,  .. .,  m 

This  condition  will  always  be  satisfied  if 

/ ot  - 1 \m  N . 

( — 5 — ^ F - 1 1'e-*  m-  , ^ 

ln 

In  the  next  section,  we  will  show  that  the  region  of  interest  for  m 
will  effectively  correspond  to  the  above  condition. 

6.5.2  Global  Optimum 

So  far,  we  have  solved  for  the  optimal  clustering  structure 
when  m,  the  number  of  levels,  is  fixed.  We  now  intend  to  let  m vary 
and,  consequently,  to  solve  for  the  global  optimum. 

Proposition  6.2 

Under  the  condition  of  Eq.  (6.20)  and  m being  a real  variable, 
the  global  optimum^  clustering  structure  is  achieved  for  a number  of 
levels 


^"Global  optimum"  is  used  to  distinguish  the  optimum  solution  with 
respect  to  m.  Also,  * indicates  values  at  global  optimality. 
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In 


N 

ff 


In 


a 

a - 1 


(6.29) 


and  a degree  vector  n* 


a 


nl*  ■ 6 a - 1 


a 


(6.30) 


^ ~ 2j  3|  . . . f m. 


k a - 1 

The  corresponding  minimum  computational  cost  is 


G*(ct,  6)  .(5-  l)ea 2l_ 

V 6 ' (a  - l)01'1 


(6.31) 


Proof: 


Since  Problem  6.11  has  been  solved  when  m is  fixed,  there  only 
remains  to  minimize  G(m,  a,  B)  over  m.  Eq.  (6.22)  may  be  rewritten  as 

_1 

G- 

where  H(m)  represents  the  expression  between  the  outside  brackets. 
Differentiating  G with  respect  to  m, 

1 


G'  » [H(m)  ] ra+1 


D*  + D , 
m+1  m+1 


f In  H(m) 

« 

L °m+l  . 

D , j^HCnOj*  = Pbh-1  ^ln  HW  - fln  H^lp;i 


m+1 


m+1 


which  we  define  as  . From  Eq.  (6.22) 


m+1 


In  H(m)  » a(a  - l)Dm  In  3^-y  - (m  - l)am  In  a 


,m-l 


+ (m  - l)(a  - l)""*  In  (a  - 1)  + am  In  N 
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Differentiating  In  H(m)  with  respect  to  m,  and  substituting  into  U,  w< 


arrive  at 


U = ot(cx  - l)(ln  6 -- ?-.■)  [D  ,0'  - D D'  ,1 
\ a - 1 ) 1 m+1  m m m+lJ 

- (m  - l)am(ln  a)  [D^  In  a - D^] 

♦ (m  - l)(a  - l)m(ln  (a-DMD^  In  (a  - 1)  - D^] 


+ a (in  N)[Dm+1  In  o - D^] 


♦ D«*i  t(°»  - D ln  («  - 1)  - a"1  In  a] 


Also,  from  Eq.  (6.23) 


D'  * a®-1  in  a - (a  - l)®'1  ln(a  - 1) 


D^+1  = a®  ln  a - (a  - 1)®  ln  (a  - 1) 


(6.32) 


Evaluating  the  terms  between  brackets  in  U,  we  obtain 


D ,D'  - D D' 
m+1  m m m+1 


„m-l . ...m-l  , a. 

a (a  - 1)  ln  q-_  { 


D ln  a - D'  , 
m+1  m+1 


- D"  *»  h4t 


Vi  In  (a  - » - Di.i 


jn  , a 

-a  ln5~rr 


Replacing  those  expressions  in  U/D^  ♦ , we  notice  that  the  last 

term  of  U will  cancel  with  D'  , and  we  are  left  with 

■T  1 


" [»(■)]  ml  a%  - 1)*  ( In  --2—)  In  (6.33) 


Since  a > 1,  G'  is  of  the  same  sign  and  has  the  sane  roots  as 

’■f(s4 r)" 

for  all  values  of  m.  Equating  the  above  equation  to  zero,  we  find  m* 
as  given  in  Eq.  (6.29).  Also,  for  n < m*,  G'  is  negative,  and  for 
m > m„,  it  is  positive.  Therefore,  we  conclude  that  G is  minimum  for 
m = m*,  and  m*  is  the  optimal  number  of  levels. 

In  order  to  find  the  corresponding  optimal  vector  n,  let  us 
notice  that  from  Eq.  (6.29) 


(6.34) 


Substituting  Eq.  (6.34)  into  Eq.  (6.21),  we  arrive  at  Eq.  (6.30). 

There  are  two  possible  ways  to  evaluate  the  global  minimum 

cost  G#. 

(i)  Replace  m by  in  Eq.  (6.22).  After  some  algebra,  usinj 
Eq.  (6.23),  Eq.  (6.22)  may  be  rewritten  as 


a i 

g = — d (e 

(a  - l)a_1  m+lV 


a(a-l)D  m 
m -ma  , 
a (a  - 1) 


\ ® ih\  m+1 

"t0-11  N°  I (6. 


(6.35) 


Then  from  Eqs.  (6.23)  and  (6.34) 


and 


Vi  ■ - 1) 

(^) 


m.am* 


Substituting  the  last  two  equations  into  Eq.  (6.35)  at  m ■ m*  , we 
get  Eq.  (6.31),  after  simplification. 
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(ii)  When  all  the  k level  clusters  are  of  equal  degree,  the 


computational  cost,  Eq.  (6.9),  becomes 


m m 


G(m,  n,  a,  B)  « £ / n n.\(B.n  ) 
k=l  ' i=k+l  V K k 


Substituting  Eq.  (6.30)  into  Eq.  (6.36),  we  find 


G*  = (n*) 


a (n*/B)  * - 1 


1 (nf/B)  - 1 

Replacing  nj  in  Eq.  (6.37)  by  B . we  get  Eq.  (6.31). 

operation  terminates  the  proof  of  Proposition  6.2. 


(6.36) 


(6.37) 


This  last 


Remarks 


1.  Size  vector:  At  global  optimality,  the  size  vector  is 


such  that 


g*  = g ~ 

gk  p a - 1 


k = 1,  2,  ...,  m* 


(6.38) 


Eq.  (6.38)  indicates  that  at  optimality,  all  the  layer  subnets  are  of 
equal  size  which  depends  only  on  a and  B.  The  explanation  of  this 
very  simple  and  interesting  property  will  be  the  object  of  Section 


6.5.3. 


2.  Gate  constraint:  From  Eq.  (6.38),  since  a > 1, 

g£  > 6 k = 1,  2,  ...,  m 

Hence,  the  gate  constraint,  Eq.  (6.6),  is  always  satisfied  at  opti- 
mality. Moreover,  as  noticed  at  the  end  of  Section  6.5.1,  the  optimal 
solution,  given  m m*,  will  also  satisfy  the  gate  constraint. 

Number  of  layer  subnets:  Eq.  (6.31)  may  be  rewritten  as 


mi  mm iClMiiliWWW 


G*  = N6 


a-l 


_2L = a JL  /6  a f 

- l)a_1  a nj  a - 1 / 


(a  - 1)' 

Since  NG*  = N/n*,  then  substituting  Eq.  (6.43)  for  k = 1 into  the 
above  equation,  we  get 


G*  - aGJ  (6.44) 

which  says  that  the  design  of  the  1S^  layer  subnets  represents 
approximately  1/a  of  the  total  computational  cost. 

5.  Behavior  of  the  optimal  tree  structure  with  respect  to  a: 
We  will  assume  that  0=1.  Let  us  define  a regular  tree  of  degree  K 
as  a tree  whose  nodes  are  all  of  equal  downward  degree  K,  except  for 
the  leaves  (downward  degree  zero),  [KNUT  69].  As  an  example,  a binary 
tree  is  a regular  tree  with  degree  2. 

The  global  optimum  solution  given  in  Eq.  (6.30)  becomes,  for 


0=1: 


k • 1.  2, 


m 


* 


(6.45) 


We  are  interested  in  the  set  of  a's  which  yield  to  integer  solutions, 
i.e.,  to  regular  tree  structures.  If  K is  the  degree  of  such  trees, 
then  a must  be  such  that 


K 


«—>  a 


(6.46) 


Moreover,  if  a regular  tree  of  degree  K is  composed  of  m*  levels,  then 
it  contains  K * leaves.  Consequently,  there  is  a one  to  one  corres- 
pondence between  the  set  of  regular  trees  of  degree  K (K  ^ 2 integer) 
whose  number  of  levels  is  m*  (integer  _>  2)  and  the  global  optimal 
solutions  of  Problem  6.11,  where  a = F K and  N = Km*  for 

K “ 1 


./!■  ...  . ..  — 
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K = 2,  3,  . Notice  that  the  above  set  of  a's  is  contained  in  the 
interval  (1,  2]  of  real  values,  i.e.,  I < a<_2.  Also,  a = 2 corres- 
ponds  to  a binary  tree  representation. 

6.5.3  Irreducibility 

The  simplicity  of  the  solution,  Eq.  (6.38),  obtained  at  global 

optimality  leads  us  to  consider  a more  intuitive  approach  toward  its 

> 

derivation,  based  on  irreducibility  considerations. 

*1 

Definition:  An  irreducible  set  of  nodes  is  such  that  no  computational 
gain  can  be  obtained  through  the  application  of  the  MIT  for  the  design 
of  the  corresponding  communication  network. 

Lemma  6.1:  At  global  optimality,  each  layer  subnet  to  be  designed  must 

correspond  to  an  irreducible  set  of  nodes. 

is 

Proof  (by  contradiction) : Assume  that  at  optimality,  there  exists  at 

least  one  reducible  unit.  Then,  the  application  of  the  MiT  to  design 
the  ne*v*ork  corresponding  to  that  specific  unit  will'  reduce  the  com- 
putational cost  for  that  unit  and  consequently  for  the  total  design. 

This  contradicts  the  fact  that  we  have  reached  the  global  optimum. 

Notice  that  the  application  of  the  MIT  to  a reducible  unit  will 
preserve  the  hierarchical  structure  of  the  network  as  defined  in  Section 
6.3.  This  would  not  be  true  if  the  a's  or  6's,  or  both,  weie  not  equal 
at  ell  levels. 

• ♦ i '*5  Our  intuition  leads  us  to  assume  that  for  a given  a and 
a*  of  an  irreducible  laiit  is  unique. 


;»i 


Fact  6.1:  At  optimality,  all  layer  subnets  must  be  of  equal  size,  q, 

v 

and  the  number  of  levels  must  be  equal  to  m*. 


q = B 


a - 1 


and  m. 


(6.47) 


a - 1 


Proof:  As  a consequence  of  Lemma  6.1  and  the  above  assumption,  at 
optimality  all  the  layer  subnets  are  of  equal  size.  Let  q be  that 
size;  then  from  Eq.  f6.5) 


(nl  = ** 

in.  = 


(6.48) 


q/  8 k = 2 , . . . , m 


From  the  size  constraint,  Eq.  (2.1) 


(6.49) 


Substituting  Eqs.  (6.48)  and  (6.49)  into  Eq.  (6.9),  we  arrive  at 


(N/8)  ' - 1 


(6.50) 


In  order  to  find  the  optimal  number  of  levels,  there  remains  to  mini- 

I 

mize  G with  respect  to  m.  Differentiating  G with  respect  to  m,  we  get 


£ ■ *°(S  - i) 


^ [■  - <•  - »(in  (•■»> 


In  practical  situations,  N/8  > 1.  Under  this  assumption,  dG/dm  is  of 
the  same  sign  and  has  the  same  roots  as  a - (a  - 1)  (N/8) 1/,m.  Equating 
this  last  term  to  zero,  we  get  m = m*,  as  given  in  Fact  6.1. 
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From  the  sign  of  dG/dm,  we  note  that  G is  minimum  for  m = m*.  This 
terminates  the  proof  of  Fact  6.1. 


Conclusion;  The  assumption  introduced  in  this  section  is  intuitively 
justified  a posteriori,  since  the  solution  of 


a - 1 

with  respect  to  N is  unique  and  equal  to  = 3a/ (a  -1).  In  other 
words,  the  optimal  number  of  levels  for  a design  of  a network  with 
3a/ (a  - 1)  nodes  is  equal  to  one. 


6.5.4  Variations  and  Limiting  Behavior  of  the  Optimal  Solution  with 


Respect  to  the  Design  Variables 

The  behavior  of  the  computational  cost  and  the  degree  vector 
at  optimality,  given  m,  will  be  studied  with  respect  to  m.  Of  inpor- 
tance  is  the  fact  that  G(m)  converges  fairly  fast  to  its  minimum  value, 
versus  m,  and  remains  very  close  to  that  value  as  m grows  to  infinity. 

A similar  phenomenon  will  characterize  the  behavior  of  the  n^'s  versus 
m.  We  will  also  notice  that,  as  a goes  to  infinity,  the  optimal 
solution,  given  m,  is  such  that  all  layer  subnets  must  be  of  equal 
size;  such  a property  was,  for  finite  a,  only  true  at  global  optimality, 
i.e. , m = m*. 

The  variations  of  G*  and  m*  with  respect  to  a,  3,  N,  do  not 
disclose  any  remarkable  property,  except  that  m*  rapidly  reaches  its 
asynptotic  value  as  a becomes  greater  than  2. 

In  what  follows,  we  will  restrict  the  study  to  the  practical 


___ 


situation  where 


6.5.6. 1 Behavior  of  the  Optimal  Solution  and  Objective  Function 


Given  m 


a.  G(m)  versus  m:  Section  6.5.2  showed  that  as  m increases 


from  zero  to  infinity,  G(ra)  decreases,  reaching  its  minimum  at  m 
(note  m#  > 0 because  of  Eq.  (6.52)  and  a > 1),  and  then  increases 
limit  of  G(m) , as  m goes  to  infinity,  is: 


A limit  G(m)  = 8 
m 


Proof 


Let  us  first  derive  some  intermediary  results.  From  Eq.  (6.23) 


Hence,  when  m goes  to  infinity 


Consequently,  substituting  these  limits  into  Eq.  (6.35),  we  arrive  at 
Eq.  (6.53).  Notice  that  the  difference 


is  independent  of  N,  and  that  the  relative  difference 


goes  to  zero  as  N becomes  large. 

In  what  follows,  four  sets  of  figures  (Figs.  6.2  - 6.9),  each 
corresponding  to  a specific  value  of  the  pair  (a,  3),  {(3,1),  (3,3) 
(5,1)  (5,3)},  will  be  shown.  In  each  set,  the  functions  G(m) 
and  G(m)/G*  are  plotted  with  respect  to  m and  for  several  values  of 
N,  N = {50,  10^,  103,  10*,  105,  10^}.  The  curves,  G(m)  versus  m, 
illustrate  the  initially  decreasing,  then  slightly  increasing  and 
asymptotic  behavior  of  the  optimal  computational  cost  for  a fixed  m. 

By  comparing  G(m)  to  Na,  we  are  able  to  appreciate  the  enormous  com- 
putational gains  obtained  through  the  application  of  the  MIT. 

The  curves,  G(m)/G*  versus  m,  illustrate  the  "locking"  effect 
whereby,  once  G(m)  reaches  its  minimum  value  of  m = m*,  Eq.  (6.29),  it 
will  appear  as  if  it  remains  indefinitely  at  that  value.  They  also 
illustrate  the  fairly  fast  convergence  of  G(m) toward  a value  close  to 
the  minimum,  for  a value  of  m relatively  smaller  than  m*.  This  indi- 
cates that  we  may  actually  obtain  most  of  the  computational  gains  with 
hierarchical  structures  whose  number  of  levels  (m)  is  much  smaller 
than  the  optimal  ones  (m„) . Finally,  the  four  sets  are  present  in 
order  to  indicate  the  effect  of  the  design  parameters  a and  6 on  the 
behavior  of  the  aforementioned  functions. 

b.  n^  versus  m 

Differentiating  Eq.  (6.21)  with  respect  to  m,  we  find,  after 
simplification 
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, , N , a 

! . in  _ + m in  __ 


^k  (a  - 1)m~kam+k"l 

dm  ' nk  „2 


D' 


m 


m+1 


For  a > 1,  dn^/dm  is  of  the  same  sign  as  the  expression  in  brackets  (to 


be  denoted  by  Z) . From  Fig.  6.10,  we  notice  that  Z has  a unique  root. 


mQ,  which  is  such  that 


m*  — mn  — m*  + 


Also,  for  m < m^,  Z is  negative  and,  conversely,  for  m > mQ,  Z is 

positive.  Z is  equal  to  zero  at  a value  of  m , m^,  which  is  independent 

of  k.  Consequently,  as  m varies  from  zero  to  infinity 

(i)  all  n^'s,  k < m^,  decrease,  reach  a minimum  and  then 
increase  toward  their  limits. 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  «UMAU  Of  STANOANOl  - l*AS  - A 


(ii)  all  n^'s,  k > mQ,  increase  toward  their  limits. 
The  limit  of  nk  for  k fixed,  as  m goes  to  infinity,  is 


limit  n = g 

_ . l ot  - 1 
m ■+■  00 


limit  nk  = — ^ k ^ 2,  k fixed 

m ■+  00 


The  proof  is  immediate,  after  observing  that 


, nm-k  k-1 

limit  m * limit 

m *►  00  m+1  m -*•  ® 


(^) 

> - (^) 


and  then,  substituting  this  limit  into  Eq.  (6.21).  If  we  let  k vary, 
more  particularly,  if  we  let  k = m,  then 


Hence 


limit  + « 

m -*•  00  m+1 


limit  n = 0 

m - « ■ 


Moreover,  In  nm  has  as  an  asymptote  a straight  line  of  equation 

m a . a / 

- a ln  + ln  (f ) 

The  behavior  of  nk  versus  m,  for  k <.  m«,  exhibits  the  "locking"  effect 
as  previous  described.  Moreover,  this  phenemonon  is  quite  remarkable 
here,  since  the  limit  of  nk,  as  m goes  to  infinity,  is  equal  to  the 
value  of  nk  at  global  optimality.  Fig.  6.11  illustrates  the  above 
properties. 
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6. 5. 4. 2 Behavior  of  G»  and  m»  with  Respect  to  a,  3,  N 


a.  G*  versus  a:  Differentiating  Eq.  (6.31)  with  respect  to 


a,  we  find 


dG*  / N 

dS“"  U 


(I  - ^ — y in  3 ~~r 
' F ' (a  - l)01"1  a - 1 


Under  the  conditions  of  Eq.  (6.52)  and  a > 1,  G*  is  an  increasing 
function  of  a.  As  a goes  to  infinity,  G*  will  be  asymptotic  to  the 
expression,  (N/3  - l)e(a  - 1/2)3°. 

If  3=1  the  asymptote  is  a straight  line,  which  implies  that 
at  the  limit,  G*  will  show  a linear  growth  with  a. 

b.  G»  versus  N:  Eq.  (6.31)  shows  that  G*  varies  linearly  with 

N.  It  is  also  important  to  notice  that  the  optimal  hierarchical  struc- 
ture reduces  the  conputational  cost  from  the  order  of  Na  steps  to  the 
order  of  N steps! 

c.  G»  versus  3:  Differentiating  Eq.  (6.31)  with  respect  to 

3,  we  arrive  at 

dG*  aa  fla-2  ,,  , 

dg-  = 7 6 l(a  ■ 1)N  • ae]- 

(a  - 1) 

Consequently,  G*  is  an  increasing  and  then  decreasing  function,  as  3 
varies  from  1 to  N (for  N > -a-  y and  under  the  conditions  of  Eq. 

(6.52)).  Equating  the  derivative  to  zero,  we  find 


R - 0-1  v 

6o  * 


(6.55) 


Substituting  Eq.  (6.55)  into  Eq.  (6.31),  we  arrive  at  the  maximum  value 
of  G*  (with  respect  to  3),  G*(3  = 3q)  ■ N°.  This  maximum  cost  corre- 
sponds to  a design  with  no  hierarchical  structure.  This  last  equation 


1 -v 
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checks  with  the  fact  that  for  a set  a,  6,  N,  satisfying  Eq.  (6. 55),  the 
optimal  number  of  levels,  m*,  is  equal  to  one  (Eq.  (6.29)),  i.e.,  for 
such  a set,  a non-hierarchical  design  is  optimal. 

d.  m»  versus  N and  6:  Recall  Eq.  (6.29), 

®*  ■ (In  N/0)/(ln  a/ (a  - 1)).  Hence,  m*  varies  as  a logarithm  of  N/B. 
This  logarithmic  behavior  appears  to  be  characteristic  of  hierarchical 
structures  [(HOW  74],  (see  Chapter  2) . 


e.  m»  versus  a:  Differentiating  Eq.  (6.29)  with  respect  to 


a,  we  find 


dm*  In  N/B 

■y  ■ s ■-—■■■■Hi  ■ i - ■ i 

a(a  - l)[ln  (a/(a  - l))]2 

Consequently,  for  a > 1,  m,  is  an  increasing  function  of  a.  The 
limiting  values  of  m are 

( ®*  -*-0 
a -►  1+  —>  | 

CO 

da 


a ♦ <*>  — > 


m*  * 00 


!n  N/B 


The  asymptote , as  a goes  to  infinity,  is  the  straight  line 

»*  - (a  - j)  In  N/B 

Fig.  6.12  shows  the  plots  of  m*  versus  a for  several  values  of  N/B. 
Notice  that  m*  rapidly  reaches  its  asymptotic  value  as  a becomes  greater 
than  2.  Also, 


1 for  a 
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which  means  that  for  1 < a < f » 

furthermore,  a loss  will  be  incurred  if  we  try  to  do  so 


no  partitioning  is  required 


6. 5. 4. 3 Limiting  Behavior  of  the  Optimal  Solution.  Given  m,  with 


we  obtain 


In  other  words,  since  the  limit  of  n is  zero,  for  any  given  m,  m ^ 2 


then,  no  clustering  must  be  allowed.  This  confirms  the  earlier  result 


obtained  when  looking  at  the  behavior  of  m„  with  respect  to  a 


limit  — 
a -*•  ® mat1 


Hence 


Consequently,  as  a goes  to  infinity,  the  optimal  solution,  given  any  m 
is  such  that  all  layer  subnets  are  of  equal  size.  Such  a property  was 
found  to  be  true  for  finite  a,  only  at  global  optimality. 


6. 5. 4. 4 Optimal  Clustering  when  a <.  1 


Notice  that  if 


1 


then,  from  Eq.  (6.29),  we  conclude  that  m*  <_  1,  which  means  that  no 
clustering  is  needed  for  a satisfying  the  equation  above.  Let  us  show 
that  this  result  holds  true  if  0 < a < 1, 

Proposition  6.3 

Ikider  the  condition,  0 < a _<  l,  the  global  optimum  of  Problem 
6.11  (with  equal  ex's)  is  achieved  for  m = 1,  i.e.,  no  clustering  is 
required. 


Proof: 

By  contradiction,  assume  that  m > 1;  then,  similar  to  the 
proof  of  Proposition  6.1  (Appendix  C) , let  us  fix  m (m  > 1,  integer) 
and  all  degrees,  nk(im,  ....  ik+1),  (k  = 2,  . . . , m) , and  solve  the 
reduced  problem  with  respect  to  nJ(im,  ....  i^'s.  i.e.. 


NC, 


min:  G = £ [n.(i)]a 

1 i*l  1 


over:  n^i) 

NCi 

s.t. : £ n (i) 

i=l  1 


= N 


(see  Eq.  (6.8)) 


(see  Eq.  (2.1)) 


n^(i)  0 real  variable 

where  NCj  denotes  the  number  of  1st  level  clusters,  and  nJ(i)  denotes 
the  size  of  an  arbitrary  1st  level  cluster.  Since  a < 1,  then  (n1(i) ]a 
is  a concave  function;  hence,  the  objective  function  above,  Gj,  is  a 


* 


! 

i 

i 

! 


concave  function.  Also,  the  set  of  feasible  vectors,  nj  = (n^i)}  is 

a bounded  convex  polyhedron  whose  vertices  correspond  to  vectors  with 

all  zeros  except  for  one  component  equal  to  N,  i.e.,  (N,  0,  0,  ....  0) 

and  its  permutations.  As  a result,  we  are  faced  with  the  optimization 

of  a concave  function  over  a bounded  convex  polyhedron,  whose  optimal 

solution  is  well  known  to  be  at  a vertex.  At  any  vertex,  = Na; 

consequently,  it  is  the  minimun.  Notice  that  this  result  is  true  for 

any  NCj,  i.e.,  for  any  nk(im,  ....  ik+1),  (k  = 2,  ...»  m) . In  other 

words,  for  any  vector  n satisfying  the  size  constraint,  Eq.  (2.1), 
a 

Gj  > N , which,  combined  with  Eq.  (6.9),  gives 

a m 

G(m,  n,  a,  g)  ^ N + £ G,  (m,  n,  a,  g)  ¥ n,  feasible 

~ ~ ~ k=2  K ~ ~ ~ 

Hence,  G(m,  n,  a,  g)  > Na  ¥ n,  feasible 

Consequently,  the  optimal  solution  of  Problem  6.11,  for  integer  m > 1, 
must  also  satisfy  the  above  inequality  which  is  a contradiction,  since, 
if  m » 1,  G(m,  n,  ot,  g)  = Na. 

Remark 

Proposition  6.3  holds  true  for  the  general  case  where  the 
o^'s  are  arbitrary,  if  < 1. 

6.5.5  Comparison  of  the  Optimal  Solution  with  Two  Feasible  Solutions 
TVo  intuitive  solutions  emerge  when  dealing  with  hierarchical 
structures.  The  R-solution  (R  stands  for  root)  corresponds  to  a regular 
tree  representation,  i.e.,  all  degrees,  at  all  levels,  are  equal. 

r » ^ k a 1,  2,  ....  m (6.57) 


4 
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The  Q-solution  models  the  situation  where  all  layer  subnets  are  of 
equal  size. 


Notice  that  the  two  solutions  are  identical  when  8=1.  The  optimal 
solution,  given  m,  will  be  referred  to  as  the  G-solution. 


6. 5. 5.1  The  Q-Solution 


This  solution  has  been  studied  in  detail  in  the  course  of  the 
proof  of  Fact  6.1  (Section  6.5.3).  Of  significance  is  the  property 
that  the  Q-solution  satisfies  Proposition  6.2  at  global  optimality. 

The  corresponding  computational  cost  (denoted  by  Q)  is  given  by  Eq. 
(6.50).  There  remains  to  find  the  limit  of  Q as  m goes  to  infinity. 
From  Eq.  (6.50), 


limit  Q = + oo 
m 

The  corresponding  asynptote  is  a straight  line  whose  equation  is 


6. 5. 5. 2 The  R-Solution 


From  Eq.  (6.57)  and  the  size  constraint,  r 
assignment,  the  conputational  cost  is  r* 


With  this 


Differentiating  Eq.  (6.60)  with  respect  to  m,  we  find 


dR  . N(“-1)/m  in  N f a ,,,.1/n 

* ' 7ZP*  - n2  i* (N ' 1,N 


m‘(N*'  ■ - l)‘  L 

- (a  - l)(N1/m  - 1)  ( b“(N  - N1/m)  + N(N1/m  - 1))J  (6.61) 
If  we  let  Y represent  the  expression  in  brackets  and  X = N1/,m,  then 
Y = -(a  - 1)  (N  - g“)X2 

+ (2(a  - 1)N  + ga(2N  - a(N  + 1)))  X + (a  - 1)N(b“  - 1) 


dR  . 

is  of  the  same  sign  as  Y.  Notice  that 


(6.62) 


X ® “>  Y **■  - <*> 


X = 1+  =>  m ->  + «=>  Y = (N  - 1)8“  > 0 (6.63) 


X =>  m -►  0 — > Y ->•  - 


Th'us,  there  exists  a unique  root,  XR,  which  is  greater  than  one.  Let 
■R  be  such  that 


N ' R _ x =>  m In  N 
N - XR  > *R  ’ TTX 


(6.64) 


From  the  above  remarks,  we  conclude  that  R is  minimum  for  m = n^. 
There  remains  to  find  the  limit  of  R as  m goes  to  infinity.  From  Eq. 
(6.60),  limit  R = ♦ °°.  The  corresponding  asyng>tote  is 

| 4 00 

K>  m <N  * [i|-N  ■♦!+(<*-  2)BaJ.  (6.65) 


if 

fe'i 

pt-  ! 

I"  ’’  i 


6. 5.5. 3 Comparison  of  the  Three  Solutions 

Let  us  first  summarize  the  properties  of  the  three  solutions. 
1.  All  three  solutions  possess  an  optimal  number  of  levels. 
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m*  (Eq.  (6.29))  for  G and  Q,  and  mR  (Eq.  (6.64))  for  R. 

2.  For  large  values  of  m,  the  G-solution  is  asymptotic  to  a 
constant;  whereas,  the  Q and  R solutions  asymptotically  grow  linearly 
with  m.  In  other  words,  for  m _>  m*,  G,  contrary  to  Q and  R,  is  not 
sensitive  to  m. 

3.  The  G and  Q solutions  meet,  for  any  a,  at  m = m*  and,  for 
a ■+  00  , at  any  given  m. 

4.  The  Q and  R solutions  are  identical  for  8=1. 

Figs.  6.13  and  6.14  illustrate  the  above  properties.  Moreover, 
we  notice  that,  as  expected,  G is  always  smaller  than  or  equal  to  Q and 
R,  and  that,  for  8^1,  Q is  certainly  better  than  R,  for  values  of  m 
in  the  neighborhood  of  m*,  but  not  necessarily  outside. 


6.5.6  Suboptimal  Integer  Solution 

We  propose,  below,  a heuristic  algorithm  which  generates  an 
integer  solution  from  the  optimal  real -valued  solution. 


Algorithm:  Given  N,  m,  8 

1.  M «-  m,  OLD  N *■  N [save  old  values] 

2.  Compute  nm  from  Eq.  (6.21) 

ndm  «-  fn  "|  ceiling  operation 
N 

N *■  —r-  adjust  size  constraint 


m *■  m - 1 

If  m > 1 GO  TO  2. 

r i m 

Else  nd..  |N|,  ND  nd. 

i*l  1 


i K 


3.  m ♦ M,  N «-  OLD  N 


4.  DO  i ■ 2,  3,  . . . , m 


OLD  nd.  «-  nd. 

l l 


*i  FnD  ndil 


(ND)  (ndi) 


OLD  nd. 

l 


5.  Confute  corresponding  computational  cost 


m / m v 
L ( II  nd.)  (3k(nd  ) ) 
k=l  V i=k+l  V v K K ' 


The  main  idea  underlying  the  above  algorithm  is  to  always 
compute  the  degree  of  the  highest  level  cluster;  then  after  readjustment 
of  the  number  of  nodes,  N,  the  operation  is  repeated  for  an  equivalent 
problem  with  one  less  level.  This  idea  is  incorporated  in  Step  2.  The 
purpose  of  Step  4 is  to  adjust  the  integer  solution,  {nd^}  , obtained 
in  Step  2,  in  order  to  reduce  the  slack  in  the  size  constraint,  i.e., 
in  order  to  reduce  the  difference  between  ND  and  the  given  value  of  N. 

The  global  minimum  may  easily  be  obtained  when  m is  restricted 
to  a certain  range  of  values.  This  could  be  done  by  conputing  the  cost 
incurred  for  each  m (using  the  above  algorithm)  and  then  choosing  the 
number  of  levels  which  corresponds  to  the  minimum  cost. 

The  performance  of  the  above  algorithm  is  illustrated  in  the 
following  sets  of  figures.  Figs.  6.15  - 6.18.  These  figures  must  be 
compared  to  those  corresponding  to  the  real-valued  solution.  Figs. 

6.2  - 6.9.  We  nctice  that  the  plots  of  G(GD)  versus  m roughly  exhibit 
the  same  shape  as  found  for  the  real-valued  solution.  The  two  solutions 
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are  also  fairly  close  for  small  values  of  m.  Moreover,  the  algorithm 
seems  to  work  better  in  the  smaller  region  of  a,  and  it  appears  that  it 
is  not  that  sensitive  to  8.  The  plots,  GN/G*,  show  the  initial  fast 
drop  in  comp ut at ion al  cost  followed  by  a levelling  behavior  to  a value 
within  2G*  for  a * 3 and  5G*  for  a = 5,  and  which  seems  to  be  fairly 
independent  of  N. 


Plication  to  Other  Special  Cases 


This  section  deals  with  the  following  two  cases: 


(i)  uniform  design  strategy,  variable  gate  assignment,  i.e., 
0^  = 0 Vk  * 1,  ...,  m and  3j^ 1 s variables. 

(ii)  proportional  assignment  of  gates. 

This  section  is  treated  in  Appendix  C.  A summary  of  the 
results  is  presented  below. 


In  (i)  explicit  expressions  for  the  optimal  degree  vector  and 
computational  cost,  given  m,  have  been  derived.  Some  partial  results 
related  the  the  global  optimality  have  been  found  when  is  of  the  form 
ek  = Yk'28  k > 2. 

With  respect  to  (ii),  the  number  of  kth  level  gates  to  be 
selected  is  proportional  to  the  number  of  k-ls*  level  gates  from  which 
they  are  selected.  The  corresponding  solution  was  found  to  be  of  no 
practical  interest.  The  solution  is 
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Delay  Expression  for  Hierarchical  Networks 
Given  a network  with  an  m-level  hierarchical  structure,  the 
problem  is  to  express  the  total  average  delay  in  the  network  in  terms 
of  the  average  delays  in  the  layer  subnets  composing  the  network.  We 
will  assume  that  all  k**1  layer  subnets  are  of  equal  size,  i.e., 

"k^m'  im-l*  ****  *k+l^  = nk  Vk  « 1,  ...,  m 

and  that  they  will  induce  the  same  average  delay,  T^,  over  the  traffic. 
This  last  assumption  will  usually  appear  as  a design  constraint,  and  as 
such,  it  is  a reasonable  one. 

The  traffic  in  hierarchical  networks  may  be  divided  into  m 
classes.  Class  k traffic  is  defined  as  the  traffic  between  pairs  of 
nodes  which  belong  to  the  same  k*^1  level  cluster  but  not  to  any  lower 
level  clusters.  If  we  define  as  the  average  delay  incurred  by  class 
k traffic,  then  obviously 

■ r. 

T * £ -fr  T.  (6.66) 

k*l  1 K 

where  = total  class  k traffic 

T,  T are  as  defined  in  Eqs.  (4.1)  and  (4.2) 

From  Flow  Assunption  6.1,  we  know  that  class  k traffic,  when  going  from 
its  origin  to  its  destination,  has  to  go  up  through  k-1  layers  to  the 
k**1  common  layer.  Then  it  will  go  down  k-1  layers  in  a manner  described 
by  Fig.  6.19. 
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LEVEL 


DELAY  DEGREE 


Figure  6.19.  Illustration  of  the  Tree  Path  of  class-k  Traffic. 


Consequently, 


Tk 3 2 £ Ti  * Tk  k > 1 


(6.67) 


rk  may  also  be  evaluated  using  the  flow  assumption  but,  in  general,  it 
will  yield  too  complicated  an  expression.  If  we  assume  a uniform 


traffic  pattern,  i.e. , y .^  = y ¥ (j,k),  then 


rk  ■ N<\  - » 


iff. 

WA  ■ 

& 


exchanged  between  C^_j(i)  and  jO)  is,  according  to  the  previous 
definition,  a class  k traffic.  The  number  of  nodes  in  a k-lst  level 


k-1 


cluster  is  equal  to  (see  Fig.  2.5)  II  n . . Hence,  the  volume  of  traffic 


i=l 


k-1 


from  j (i)  to  C^jfj)  Is,  for  any  (i,j),  ( II  n ^ y . 


i=l 


c ♦ aL 

The  number  of  pairs  of  k-1  level  clusters  in  a kL  level 


th 


cluster  is  nk(nk  - 1);  also,  the  number  of  k level  clusters  is 


m 


n n. . Consequently,  the  total  amount  of  class  k traffic  is 
i=k+l  1 


= (ii.ni)vnk->)(  n|ni) 


k-1 


rk  = 


liJ“k(nk  ‘ 13 


(6.69) 


m 


From  the  size  constraint,  Eq.  (2.1),  we  know  that  IT  n.  = N.  Substi- 


i=l 


tuting  this  last  equation  into  Eq.  (6.69),  we  find  Eq.  (6.68). 


Notice  that  Eq.  (6.68)  checks  with  the  fact  that  £ r.  = T. 


k=l 


We  are  now  ready  to  derive  an  expression  of  T in  terms  of  the 


T 's 
k S* 


Proposition  6.4 

Under  the  above  assumptions,  the  total  average  delay  in  a 
hierarchical  network  is 

m-1  \ m-1  / k-1 


1 f/  \ / k-1  v 

rrr  [("  - kn  ♦ I.  - c-k  * <>  .n 


Proof: 


(6.70) 


Substituting  Eq.  (6.67)  into  Eq.  (6.66),  we  arrive  at 


m 


= fE(rk  + 2 £ r )t 

k=r  K i=k+i  k 


T = 


(6.71) 
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Notice  that  Eq.  (6.71)  is  a general  expression  of  T in  terms  of  T^'s 

which  does  not  use  the  uniform  traffic  assumption.  From  Eq.  (6.68), 

m k-i  v 

rk  + 2 £ I*.  - Ny  2N  - (n  ♦ 1)  II  nj  (6.72) 

K i=k+l  1 V k i=l  ' 

Substituting  Eq.  (6.72)  into  Eq.  (6.71),  we  find  Eq.  (6.70). 

Numerical  example* : 


N = 103,  nj  = n2  = n3  = 10,  6=1 

T = 0.9T3  ♦ 1.82T2  ♦ 1 . 99Tj  . 


(6.73) 


6. 8 Conclusion 

In  this  chapter  we  studied  the  major  aspects  related  to  the 
hierarchical  design  of  large  computer  networks.  The  focus  was  primarily 
upon  the  determination  of  a certain  clustering  structure  of  the  set  of 
nodes  to  be  used  in  the  design  phase.  Optimal  clustering  structures 
were  determined  so  as  to  minimize  the  computational  cost  required  in  the 
design  phase.  The  general  solution  (i.e.,  different  design  strategies 
and  gate  assignment  from  one  level  to  another)  was  derived  when  the 
number  of  hierarchical  levels  m is  fixed.  The  global  optimum  solution 
was  obtained  with  the  more  uniform  case  whereby  the  same  design  strategy 
and  gate  assignment  are  used  at  all  levels.  The  global  optimum  solution 
is  such  that  all  degrees  at  all  levels  (except  the  first  level)  are 
equal  (all  layer  subnets  are  of  equal  size).  Such  a peculiar  property 

]A  similar  expression  as  in  Eq.  (6.73)  is  given  in  [NAC  73];  It  is 
T * Tj  ♦ 2T2  ♦ 2Tj . 
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was  found  to  have  an  intuitive  explanation.  Furthermore,  we  mention 
the  one-to-one  correspondence  of  the  global  optimum  solution  for 
a = K/ (K  - 1)  with  some  regular  trees  of  downward  degree  K. 

Finally,  we  were  able  to  decompose  the  average  message  delay 
in  a hierarchical  network  in  terms  of  the  average  delays  in  the  layer 
subnets  composing  the  network. 


CHAPTER  7 


CONCLUSIONS  AND  SUGGESTIONS  FOR  FUTURE  RESEARCH 
7. 1 Conclusions 

Faced  with  the  prohibitive  cost  of  a simple  extrapolation  of 
present  design  and  routing  procedures  for  large  networks,  the  goal  of 
this  dissertation  was  to  evaluate  some  new  techniques  to  be  used  in  the 
context  of  large  networks.  The  techniques  studied  here  represent  an 
extension  of  present  schemes  and  rely  mainly  on  the  natural  hierarchical 
clustering  of  the  network  nodes.  More  specifically  we  specified,  eval- 
uated and  discussed  the  adaptive  m- level  Hierarchical  Routing  (MR) 
schemes  as  well  as  the  m-level  Hierarchical  Topology  (NUT)  design  of 
large  networks.  The  results  obtained  are  summarized  in  Section  1.3. 
Basically,  in  this  research  we  were  able  to  show  that: 

1)  Under  some  reasonable  cost  and  performance  constraints  for 
a class  of  large  distributed  networks,  present  routing  schemes  become 
infeasible,  whereas  hierarchical  routing  schemes  with  optimally  chosen 
table  lengths  maintain  remarkably  good  network  performance  for  a phe- 
nomenal range  of  network  sizes. 

2)  Some  optimal  strictures  were  found  to  minimize  the  com- 
putational cost  involved  in  the  hierarchical  design  of  large  networks. 

3)  Some  extensions  of  existing  queueing  models  for  networks 
were  introduced  in  order  to  accomodate  nodal  storage  requirements  and 
line  overhead  due  to  routing  updates  which  become  critical  in  the  context 
of  large  nets.  Furthermore,  several  buffer  sharing  schemes  were  pro- 
posed and  analyzed  in  order  to  optimally  utilize  the  nodal  storage. 


7.2  Future  Research 

While  there  exists  an  abundant  literature  on  issues  related  to 
the  design  of  small  and  moderate  sized  networks,  the  study  of  large 
networks  is  still  in  its  very  early  stages.  Consequently,  we  do  not 
only  inherit  unsolved  (or  poorly  solved)  problems  related  to  computer 
networks,  but  also  those  adequately  solved  need  to  be  reevaluated  in 
the  context  of  large  nets.  Of  importance  are  issues  related  to  the 
flow  control,  reliability,  security  and  distributed  data  bases.  We 
also  must  not  forget  the  long-standing  search  for  exact  solutions  of 
the  topology  design  problem  and  for  more  precise  analytic  network 
queueing  models. 

As  for  research  areas  which  emerge  directly  from  this  work, 
we  mention  the  important  issue  of  clustering,  i.e.,  assignment  of  nodes 
to  clusters,  clusters  to  superclusters,  etc.  We  briefly  addressed  that 
question  in  Chapter  5,  but  further  work  is  certainly  required.  Other 
clustering  techniques  (such  as  those  based  on  a graph  theoretic  approach) 
must  be  investigated  and  experimented  with.  The  same  issue  arises  in 
the  topology  design  of  large  nets  where  altogether  different  nearness 
measures  and  clustering  techniques  may  be  required.  Moreover,  the 
hierarchical  design  procedures  should  be  compared  to  the  non-hierarchical 
ones  on  some  theoretic  grounds,  rather  than  through  experimentation, 
using  those  algorithms  which  we  here  proved  to  be  extremely  expensive 
in  the  non-hierarchical  case.  We  conjecture  that  some  further  limiting 
results  of  interest  are  possible. 

Some  new  routing  schemes  and  design  methods  should  also  be 
investigated. 


| 
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Finally,  it  appears  that  the  general  methodology  and  decom- 
position models  developed  here  for  the  study  of  large  nets,  may  be 
directly  applicable  or  extended  to  more  general  large  systems  where 
some  sort  of  decomposition  must  be  introduced  to  alleviate  the  diffi- 
culty in  analysis,  design  and  evaluation.  The  identification  of  such 
systems  and  their  study  represent  a worthwhile  investigation. 
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APPENDIX  A 


PATH  LENGTH  IN  GRID  AND  TORUS  NETWORKS 


The  purpose  of  this  appendix  is  to  derive  closed  form  expres 
sion  for  the  average  path  length  in  Grid- type  networks.  Also  the 
distribution  and  the  corresponding  z-transform  of  path  lengths  in  a 
Torus  network,  are  defined  and  determined. 


A. 1 Definitions 


Let  (S,  A)  be  a network  composed  of  a set  of  nodes  S and  a set 

of  arcs  A.  Let  s,  t be  any  pair  of  nodes  in  the  network  and  * „ be 

st 

any  path  between  the  two  nodes  [HARA  72],  [HU  69].  Also  let  &i  repre- 
sent the  length  associated  with  arc  i,  then  the  length  of  a path  * 
is  defined  as. 


The  length  of  the  shortest  path  from  s to  t,  is 


hst  as  defined  ab°ve  is  a distance  function  [HARA  72],  [FRAN  71] 

If  the  length  of  all  channels  is  assumed  to  be  equal  to  1,  then 
hst  rePresents  tiie  minimum  number  of  hops  (channels)  between  s and  t. 

TTie  average  (shortest)  path  length  in  a network  is  defined  as 
[KLEI  64], 


" ’ n<n1-  » s.KS  "st 


(A.2) 


where  N is  the  size  of  the  set  S.  In  some  instances  it  is  of  interest 
to  consider  the  weighted  average  path  length,  hw. 


hw  = f £ Y th 
s.teS 


st 


(A.3) 


where  is  the  weight  on  the  pair  of  nodes,  s,  t,  and 


r A E Y 
s,teS 


(A.  4) 


Notice  that  if  all  the  Yst's  are  equal  then  Eq.  (A.3)  reduces  to  Eq. 
(A.2). 

The  diameter,  d,  of  a network  is  defined  as  the  longest 
shortest  path  in  the  network,  i.e.. 


d = max  {h  } (A.  5) 

s.teS  S 

In  the  rest  of  this  appendix  we  will  restrict  our  considerations  to  the 
hop  distance. 

A.2  Average  Path  Length  of  a Grid 

Let  G be  a rectangular  p x q grid  as  shown  in  Fig.  A. 1.  Let 
s of  coordinates  (x,  y)  and  t of  coordinates  (u,  v)  be  two  arbitrary 
nodes  in  G.  Then  from  Fig.  A. 1 we  see  that 


hst  " lu  - xl  + lv  ' yl 


(A.  6) 


Let  Hs(x,  y)  denote  the  distances  (length  of  shortest  path)  from  s to 
all  other  nodes  in  G, 
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H (x,  y)  A £ h 


(A.  7) 


Substituting  Eq.  (A. 6)  in  the  above  equation,  we  find 


P q 


Hs(x*  y)  = £ Z(|u  - x|  + |v  - y| ) (A. 8) 

u=l  v=l 


p q 

HS(X,  y)  = q £ |u  - x|  + p £ (V  - y) 
u=l  • V=1 


Hs(x,  y)  = q^x  - (p  + l)x  + + pfy^  - (q  + l)y  + £l£_L_12.j 


(A.  9) 


Eq.  (A. 2)  may  be  rewritten  as. 


r^TT  £ Hs(x-  y>  - n(n]-  i)  x4  5,  y>  «•»<» 


Substituting  Eq.  (A. 9)  into  the  above  summation,  and  after  some 
algebra,  we  arrive  at 


P q 


£ L Hs(x*  y)  = iq2p(p  +i)(p  - i)  ♦ t p2q(q  + i)(q  - i) 

x=l  y=l 


N(N  - 1)  = pq(pq  - 1) 


(A. 11) 


Hence 


(A. 12) 


From  Fig.  A.l,  we  can  immediately  see  that  the  diameter  of 


d = p + q - 2 


(A.  13) 
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For  a square  grid  p = q = \ZiT,  hence 

< d = 2 /F 


(A. 14) 


A. 3 Average  Path  Length  of  a Torus 

A torus  is  a grid  where  the  outside  nodes  are  connected 
together  in  a ring-like  fashion,  as  shown  in  Fig.  A. 2.  The  purpose  of 
the  external  rings  is  to  introduce  topological  symmetry  with  respect 
to  the  nodes.  In  other  words,  all  nodes  are  topologically  equivalent. 
As  a consequence,  the  distribution  of  path  lengths  from  a given  node 
to  any  other  node  is  the  same  for  all  nodes.  Hence  the  average  path 
length  simply  becomes. 


h ■ »rh-  £ h 


st  ■ ij-rr  HsCx-  y) 


(A. 15) 


where  s is  an  arbitrary  node  of  coordinate  (x,  y).  In  order  to  sim- 
plify the  calculations  we  choose  s as  shown  in  Fig.  A. 2.  This  figure 
shows  all  such  nodes  s,  for  all  cases  except  for  p odd  and  q even, 
which  is  equivalent  to  p even  and  q odd.  The  addition  (to  a grid) 
of  the  external  links  does  not  affect  the  confutation  of  Hs(x,  y)  for 

such  a choice  of  s;  hence  the  result  in  Eq.  (A. 9)  is  true  for  those 

/ 

nodes.  Also  the  diameter  of  the  torus  is  equal  to  the  distance  from  s 
to  the  upper  right  comer  node.  Several  cases  to  consider  are: 


i.  p,  q even 


h = n~T  HstP/2»  V2] 
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1 


Since  N = pq  and  Hg  is  given  by  Eq.  (A. 9),  we  get 

2 2 

/ . QD  + DQ 


4(pq  - 1) 


d = E-p 


ii.  p,  q odd 


h L_  H fg-t  1 

N - 1 s \ 2 * 2 ) 

h - q(p  + 1)(P  - 1)  + P(q  + 1)  (q  - 1) 


(A. 16) 


(pq  - i] 


iii.  p even,  q odd 


h ■ n4ths(!- 


p + q 

4 


(A.  17) 


+ Dfa  + 


(n  = k / v 

4(pq  - 1) 

) d = p,  + q - 1 


(A. 18) 


iv.  p odd,  q even 


4(pq  - 1) 


+ q - 1 


(A. 19) 


Square  Torus 


For  a square  torus  p = q = yJH 
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i.  p even 


E 


W$, 


( h = 1 

J 2(p^  - 1) 

( d = p = /N* 


N 


3/2 


2(N  -1) 


(A. 20) 


ii.  p odd 


h-2 


/n 


d = p - 1 = /F  - 


(A. 21) 


1 


Notice  that  for  a large  N Eqs.  (A. 20)  and  (A. 21)  are  equivalent. 
Hence  in  the  study  of  large  tori,  we  will  not  differentiate  between 
the  two  cases  above. 


A.  4 Distribution  of  Path  Lengths  in  a Square  Torus 


Let  hg  be  a discrete  random  variable  which  represents  the 


distance  (length  of  shortest  path)  from  an  arbitrary  node  s to  any 
other  node  in  the  network. 


The  probability  distribution  of  hg  is  defined  as. 


p fK  vi  number  of  nodes  at  a distance  k from  s 
rl~s  J N - 1 


The  corresponding  z-transform  is  defined  as. 


Hs(2)  = £ Z Pr[!ls  = 


(A.  23) 


Similarly,  let  h be  the  random  variable  which  represents  distances  in 

«v 

the  network.  The  probability  distribution  of  h is  defined  as 
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pr[h  = k]  = 


number  of  pairs  of  nodes  separated  by  a distance  equal  to  k 


N(N  - 1) 


Also  the  corresponding  z transform  is 

r _k 


H(z)  = l zKP  [h  = k] 
k r ~ 


(A. 24) 


(A. 25) 


Evaluation  of  H(z)  for  a square  torus 


Since  a torus  is  symmetrical  with  respect  to  any  node,  then. 


hs  = h =>  Hs(z)  = H(z)  ¥ s e S.  (A. 26) 


Consequently  we  will  evaluate  the  transform  at  a particular  node  s. 
When  p = /n~ is  odd,  the  evaluation  of  Hg(z)  is  straightforward  at  our 
previous  node  s.  Fig.  A. 3 shows  node  s and  contours  of  nodes  at  equal 
distance  from  s . 

CONTOUR  OF  NODES  AT  THE  SAME  DISTANCE  FROM  s.  y 

v 7 
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Figure  A.3.  Square  Torus. 


- W'MW  . « »m  n ii»  n*-» 


If  we  define  as  the  number  of  nodes  at  distance  k from  s,  then 


4(p  - k) 


1 £ k £ *— y 


+ 1 < k < p - 1 
2 ~ ~ 


Note' that 


P-1  2 

E N = p - 1 = N - 1 
k=l  K 


From  the  definition  of  the  distribution  of  path  length,  Eq.  (A. 22) 


(r 

yh-k]  * 

VO 


1 < k < p - 1 


otherwise 


Substituting  Eq.  (A. 29)  into  Eq.  (A. 25),  we  arrive  at 


p-1  N. 


«(*)  - L 

k«l 


k k 


and  from  Eq.  (A. 2 7) , 


N ^ 1 l ktl 


-l)/2  k P-1  j 

L kz  + L (p  - k)z 

k=l  k= (p+l)/2 


To  evaluate  the  above  summations  we  need  the  identity  below 


£ kz^  = — - — rr  (1  + nzn+*  - (n  + l)zn) 
k=l  (1-z) 


Then  after  some  algebra,  we  find 


4z(l  - ztP-^Kl  - z^1"2) 


(N  - 1)(1  - z)‘ 


(A. 31) 
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Remarks 


i.  From  the  definition  of  the  z- transform  (Eq.  (A. 23))  H(z) 
must  be  such  that  H(l)  = 1.  This  fact  can  be  easily  checked  using 
l'Hopital's  rule. 

ii.  H'(l)  represents  the  average  of  the  random  variable  h. 

It  corresponds  to  the  actual  average  path  length  (in  hops)  in  the  net- 
wark.  Hence,  from  Eq.  (A. 21),  H(z)  must  be  such  that 

H'(l)  = h = p/2  (A. 32) 

Differentiating  Eq.  (A. 31)  with  respect  to  z,  we  find 

H'(z)  = =•  fl  + z - (p  - l)zp+1  + (p  + 1)  zP  - 3z^P+1^2 

(N  - 1)  (1  - z)5  L 

+ P -_1  z(P+3)/2  _ p + 1 z(p-l)/2] 

The  application  of  l'Hopital's  rule  to  H'(z)  at  z = 1,  shows  that 
effectively  Eq.  (A. 32)  holds  true. 

iii.  For  p even,  Eq.  (A. 31)  is  no  longer  true.  However,  for 
a large  N (as  is  the  case  in  Chapter  4)  we  may  confidently  use  Eq.  (A. 31) 


in  both  cases. 


APPENDIX  B 


ANALYSIS  OF  SHARED  STORAGE 
IN  A COMPUTER  NETWORK  ENVIRONMENT 

Our  earlier  considerations  (Chapter  4)  lead  us  to  model  the 
store-and- forward  (S/F)  function  of  a node,  as  R M | M | 1 queueing  systems 
which  share  a finite  waiting  room,  under  some  scheme  (see  Fig.  4.12). 
The  purpose  of  this  appendix  is  to  analyze  and  compare  a few  existing 
and/or  intuitive  schemes  as  well  as  others  motivated  by  the  limitations 
(or  deficiencies)  of  the  former. 

The  first  (and  simplest)  scheme  is  the  Complete  Partitioning 
(CP)  scheme  where  actually  no  sharing  is  provided,  but  where  the  entire 
storage  (waiting  room)  is  partitioned  among  the  R servers  (see  Fig. 
B.l.a).  At  the  other  extreme  is  the  second  scheme.  Complete  Sharing 
(CS) , which  is  such  that  an  arriving  customer  is  accepted  if  space  is 
available,  regardless  of  its  class,  i.e.,  independently  of  the  server 
to  which  it  is  directed  (see  Fig.  B.l.b).  CS  succeeds  in  achieving  a 
better  performance  than  CP  (smaller  probability  of  blocking)  under 
normal  traffic  conditions  and  for  fairly  balanced  input  systems.  How- 
ever, for  quite  asymetrical  input  rates1  (A.^  i = 1,  . . . , R) , CS  tends 
to  heavily  favor  servers  with  higher  input  rates,  even  though  they  may 
be  close  to  saturation  (input  rate  equal  or  almost  equal  to  service 
rate).  The  failure  to  recognize  servers  at  or  near  saturation  results 
in  most  of  the  space  being  occupied  by  customers  waiting  for  those 

1This  remark  assumes  that  all  servers  have  equal  service  rates. 
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servers,  at  the  detriment  of  the  others.  Moreover,  even  with  perfectly 
balanced  arrival  rates  (i.e.,  X^  = X i = 1,  ....  R),  under  heavy 
traffic  conditions  (X  » yC) , CS  fails  (where  CP  succeeds)  in  securing 
a full  utilization  of  all  the  R servers. 

The  above  considerations  intuitively  indicate  that  contention 
for  space  must  be  limited  in  some  ways.  In  order  to  avoid  the  possible 
utilization  of  the  entire  space  by  any  particular  type(s)  of  customers, 
we  impose  a limit  on  the  number  of  buffers  to  be  allocated  at  any  time, 
to  any  server.  This  idea  is  incorporated  in  our  third  scheme:  Sharing 

with  Maximum  Queues  (SMXQ) , Fig.  B.l.c.  Of  course,  the  sum  of  those 
maxima  must  be  greater  than  the  total  space  if  some  sharing  is  to  be 
provided.  The  SMXQ,  however,  does  not  guarantee  a full  utilization  of 
‘the  servers  under  heavy  traffic  conditions.  This  deficiency  motivates 
the  fourth  scheme:  Sharing  with  a Minimum  Allocation  (SMA)  scheme. 

With  SMA,  a minimum  number  of  buffers  is  always  reserved  for  each 
server  and,  in  addition,  a common  pool  of  buffers  is  to  be  shared  among 
all  the  servers,  with  no  constraints  on  the  queue  size  (see  Fig.  3.1.d). 

The  shared  area  is  again  prone  to  be,  as  mentioned  earlier, 
unfairly  utilized,  and  hence  the  fifth  and  final  scheme:  Sharing  with 

a Maximum  Queue  and  a Minimum  Allocation  (SMQMA) (see  Fig.  B.l.e). 

Rich  and  Schwartz  [RICH  75]  studied  a scheme  very  similar  to 
SMA  except  that  the  entire  common  storage  is  dynamically  allocated  to 
one  server  at  a time.  Moreover,  problems  of  this  sort  are  frequently 
encountered  in  telephony  and  are  referred  to  as  graded  systems  [KUHN  73]. 
[SYSK  60].  Their  main  interest,  however,  is  in  sharing  (extra)  lines 
as  opposed  to  storage. 
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c.  SMXQ 


^ bi 


d.  SMA  e.  SMQMA 

Figure  8.1.  Storage  Sharing  Schema*. 
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In  this  appendix,  we  intend  to  characterize  the  five  schemes 
under  steady  state  conditions;  namely,  we  derive  expressions  for  the 
probabilities  of  blocking,  the  joint  and  marginal  distributions,  the 
average  time  in  system  and  the  throughput  for  different  types  of  cus- 
tomers. The  key  to  the  analysis  lies  in  the  fact  that,  in  steady  state, 
the  joint  probability  distribution  obeys  the  well  known  product  form 
solution  for  networks  of  queues  [JACK  57,  63],  [GORD  67],  [BASK  75], 
[BUZE  71],  [WONG  75],  [KLEI  75,  76]  (and  the  bibliographies  therein). 

A conparison  of  the  sharing  schemes  is  also  provided. 

B.l  Complete  Partitioning  (CP) 

With  CP,  we  are  in  the  presence  of  R separate  and  independent 
M|M| 1 queueing  systems  with  finite  waiting  rooms  (Fig.  B.l. a).  Queueing 
system  i (i  = 1 , . . . , R)  is  characterized  by  a Poisson  input  stream  of 
rate  A^,  an  exponential  service  time  of  mean  1/pC  and  a finite  storage 
of  size  b^.  Customers  destined  to  server  i are  referred  to  as  type  or 
class  i customers.  Arriving  customers  who  find  no  space  in  their 
respective  queues  depart  without  service.  Accepted  (non  rejected) 
customers  are  served  on  a First -Come -First -Serve  (FCFS)  basis. 

The  basic  equations  describing  the  behavior  of  any  of  the 
queues  (say  subsystem  i)  are  well  known  (see  for  example,  [KLEI  75]). 

Let  n^  be  the  number  of  customers  in  queueing  system  i (i.e.,  type-i 
customers)  and  Pr[n^  = k]  be  the  probability,  in  steady  state,  of  having 
k type-i  customers  (in  queue  and  server) , then 


Pr[n.  = k]  = 


- pi  k 
b^+l  pi 

Pi' 


0 1 k 1 bi 


(B.l) 


otherwise 


Where  pt  = X^uC. 


Also,  let  PBi  denote  the  probability  of  blocking  for  type-i  customers 
then  [KLEI  75] 


1 - p b 
PBi b^  piX 


(B.  2) 


! - Pi1 


Let  m be  the  average  number  of  type-i  customers;  then 
b. 

i p. 

n.  = £ k P [n.  = k]  = ^ — i— 

1 k-1  r 1 1 - Pi 


b.  1+b. 

Pi  1 - (1  ♦ b.Jp.1  + b.p.  1 

l+b. 

1 - P,  1 


(B.  3) 


Let  be  the  average  rate  of  non-re jected  type-i  customers,  i.e.,  th< 
throughput  of  server  i;  then 


x:  = (1  - PB. ) X. 

i i/  i 


(B.4) 


Also  if  ti  denotes  the  average  time  in  system  (queue  and  server)  of  nc 
blocked  type-i  customer,  then  from  Little's  result 


ti  - n./xr 


b.  1+b. 

1/pC  1 - (1  ♦ b.jp.1  4 b.p.  1 


(B.  5) 


. 


The  state  of  the  entire  system  can  be  described  by  the  vector 
n = (nj,  n 2,  ....  nR);  since  all  the  subsystems  are  independent,  the 
joint  probability  distribution  in  steady  state  (to  be  denoted  by  P(n) 
or  PCnj,  n2,  . nR))  is 


R /Rl-p.  vRn. 

P(nr  n2,  ....  nR)  = JI  Pn  - ( .11  I7b7  ) J1  piX 

i=l  1 ' i=l  . l / i*l 

1 - Pi 


¥ n e F 


(B.7) 


= 0 otherwise 


Where  the  set  F is  the  set  of  feasible  vectors, 

a 

Ffl  = {n  integer  valued  vector,  s.t.  0 nA  < b.,  i = 1,  ....  r} 

(B . 8) 

The  subscript  "a"  is  used  here  to  remind  us  of  scheme  (a),  i.e.,  CP. 

Notice  that  the  constant  term  in  Eq.  (B.7)  is,  of  course,  the 
probability  of  the  entire  system  being  empty,  i.e.,  P(0,  ...,  0).  That 
term  will  be  denoted  in  the  sequel  as  P^. 

This  terminates  the  analysis  of  CP  in  its  general  form.  A few 
more  remarks  are  presented  below. 

i.  bi  -*■  oo  and  < 1 ; we  obtain  the  usual  Mj  MJ 1 result 


Prln±  = k]  = (1  - p.)p* 


(B.  9) 


Pj  ■+  00  ; from  Eqs.  (B.l)  and  (B.2),  we  get 


PB.  -►  1 

l 

"T  -bi 
K -*,c1 

h ■*  bil,ci 


(B. 10) 
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iii.  » 1;  all  states  * 0,  1,  are  equally  probable 


PB.  = t - . 
l 1 ♦ b. 

l 

b. 

l 

ni  * T 


(B. 11) 


We  now  proceed  with  the  analysis  of  CS.  The  definitions  and 
notation  introduced  in  this  section  are  valid  throughout  this  appendix. 


B.2  Coaplete  Sharing  (CS) 

We  now  combine  all  the  individual  waiting  rooms  in  a global 
one  (see  Fig.  B.l.b)  whose  size  will  be  denoted  by  B.  Empty  space  is 
allocated  on  a FCFS  basis  regardless  of  the  type  of  arriving  customer. 
Drukey  [DRUK  75]  analyzed  this  system  with  the  assumption  that  all  the 
Pi's  (P^  = are  equal;  for  the  general  case  of  different  p^'s, 

he  restricts  his  study  to  two  classes  of  customer  (R  = 2).  In  what 
follows  we  intend  to  analyze  the  more  general  case  of  arbitrary  p^'s 
i = 1,  ....  R,  and  then  we  apply  our  results  to  the  special  case  of 
equal  p^'s.  The  forms  of  the  expressions  in  common  with  Drukey's  differ 
because  of  the  network  of  queues  approach  used  here. 

B.2.1  General  Case 

In  this  section,  the  p^'s  are  not  necessarily  equal.  The 
sharing  of  space  introduces  dependencies  among  the  R queueing  systems. 
The  entire  system  is  a birth-death  process  whose  state  can  be  simply 
described  by  the  vector  n = (nj,  . . . , nR) . From  the  state-transition- 
rate  diagram  of  Fig.  B.2,  we  can  write  the  set  of  balance  equations 
below,  describing  the  behavior  of  the  system  in  steady  state  (steady 
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state  is  always  reached,  even  for  > 1,  because  of  the  finite  number 
of  states  in  our  ergodic  Markov  chain) . 


IF  n«  >1\  / — TRANSITION  ALLOWED  IF  n:  > 1 


only  if  nR  > 0 TRANSITION  ALLOWED  only  if  Snj  < B 


Figure  B.2.  State-Transition-Rate  Diagram. 


Let  J = {j  such  that  n^  = 0} ; two  sets  of  balance  equations  are 

presented  below  which  depend  on  whether  or  not  the  total  number  of 

customers  is  less  than  or  equal  to  B. 

i.  For  all  n*s,  such  that  Z.n.  < B,  we  have 
~ i l ’ 

R R 

X,  •••»  n^~l*  •••*  n^)  + p^P(nj,  n^+1 , n^) 

i^J 

/ R R v 

= IS  ^ + hi  yi)P(ni ”i 

i$J 


I 


■ 


ii.  For  all  n's,  such  that  E.n.  = B, 
~ 1 1 ' 


Let  be  the  state  of  feasible  states,  i.e.,~ 

R 

Fb  = {n,  s.t.  0 < ni  < B , 0 < ^ n.  <_  B} 

i=l 

Then,  we  claim  that  the  steady  state  joint  queue  length  (queue  + server) 
distribution  is  such  that 

nl  n2  nR 

( P0P1  p2  ***PR  ¥ ~ e Fb 

P(n)  =<  (B.  12) 

{ 0 otherwise 

where  PQ  is  a constant;  more  precisely,  it  is  the  probability  of  an 
enpty  system  P(0,  0,  ....  0). 

To  prove  this  claim  it  is  sufficient  to  show  that  Eq.  (B.12) 
satisfies  the  above  balance  equations;  this  is  easily  shown  to  be  the 
case.  Notice  also  that  the  product  form  solution  satisfies  the  local 
balance  equations.  In  the  present  situation,  the  local  balance  equation 
[CHAN  72],  [KLEI  75]  is  the  one  which  equates  the  rate  of  flow  out  of 
server  i,  due  to  the  departure  of  customer  (type-i) , to  the  rate  of 
flow  in,  due  to  the  arrival  of  a customer,  to  server  i.  This  approach 
will  be  used  to  analyze  the  other  schemes  whose  global  balance  equations 
are  more  difficult  to  write. 

We  are  now  left  with  the  evaluation  of  P^,  for  that  we  take 
advantage  of  the  fact  that  all  probabilities  must  sum  to  one,  hence 
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nl  n2  nR 


P0  = P1  P2  •*,pR 


(B. 13) 


Let  us  define  G(K)  as 


ni  "r 


G(K)  = Z P 1 ..p  ' 

xl/  ^ 


a<n . <K 

— l— 

E.n.=K 

l l 


Then,  obviously 


Pn1  = L GCK) 
K=0 


(B. 14) 


(B. 15) 


Several  efficient  altorithms  exist  to  compute  G(K),  [BUZE  71],  [WILL  74] 
[MDOR  72].  Moreover,  if  all  the  p^'s  are  different,  we  can  use  the 
generating  function  approach  to  derive  a closed  form  expression  for 
G(K) , [MOOR  72],  Briefly,  we  let 

R 1 R 

g(t)  = H 7f  - p t)  = II  (1  + P.t  ♦ p?t2  ♦ •••)  (B. 16) 

i=l  v Mi  1 i=l  x 1 

By  expanding  the  second  product  we  recognize  that  G(K)  is  the  coefficient 


- . K . 
of  t , i.e. 


g(t)  = 1 + G(l) t + G(2)t2  ♦ •••  ♦ G(K)tK  + 


(B. 17) 


Also,  if  the  p^s  are  different1  (pA  t p..),  then  the  part:>l  fraction 
expression  of  the  first  product  in  Eq.  (B.161  gives 


R , R A. 

( i?i  (1  - V5  ’ i?i  1 - v 

) R i 

I where  A.  = TT  — 7 — r 

1 1 k- 1 


(B.i8) 


TTiis  condition  will  be  assigned  to  be  true  throughout  the  rest  of  this 
section. 
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Hence 


Eqs.  (B.12)  and  (B.20)  completely  characterize  our  system  in 


Notice  that  for  R = 1,  A 


We  now  proceed  to  derive  the  distributions 


or  expressions  of  other  variables  of  interest 


Distribution  of  the  total  number  in  system;  Probability  of  Blockin 


Let  n be  that  number  and  P 


n]  be  the  corresponding 


distribution;  then  obviously 


otherwise 


Since  we  have  Poisson  arrivals,  then  the  probability  of  blocking,  PB, 
is  simply  the  probability  that  n = B.  Notice  also,  that  PB  is  indepen 
dent  of  the  customer's  type  (PB.  = PB  ¥ i = 1,  ...,  R), 


Marginal  Distributions  and  Averages 


Let  us  first  derive  the  probability  that  there  are  more  than  j 
type  i customers  in  the  system,  pr[n^  > j].  That  probability  is  equal 
to  the  sum  of  P(n)  for  n e F^  and  such  that  j;  after  some  algebra 
we  find  (for  0 _<  j £ B) , 


i B"j 

PrIni  > j]  * p0Pi  L G(n) 
n=0 


(B. 23) 


The  average  number  of  type  i customers  is 

B b B- j 

"i  = L pr[ni  iil  = p0  I p{  L G(n) 

j=l  j=l  n=0 

Interchanging  the  summations  above,  we  arrive  at 

B-l  B-n  . B-l  p.  - pB+1_n 

5i  - p0  E W>  E Pi  - p0  En  -T-rr-  «W 

n=0  j=l  n=0  l 


hence 


— i n=0 
"i  = — - 


E (1  - P-'n)G(n) 


E G<") 

n=0 


(B. 24) 


As  for  the  expression  of  the  average  total  number  in  system,  it  is 
simply  n = EJk.  Another  expression  can  be  derived  from  Eq.  (B.21), 


n = pn  Z kG(k) 
u k=l 


From  Eq.  (B.19) 


" = po  £ Ai 


1 + Bpj+B  - (B  ♦ 1)PB 


where  F-  is  given  by  Eq.  (B.20). 


(1  - P^' 


(B. 25) 
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Notice  that  if  R = 1,  then  n = n^,  as  found  in  Eq.  (B.3).  Eqs.  (B.4) 
and  (B.5),  for  the  throughput  A^  and  the  average  delay  t^,  hold  true 


here. 


This  terminates  the  characterization  of  CS  in  its  more  general 


case.  Of  interest  is  the  study  of  its  behavior  under  some  special 
limiting  conditions  of  storage  and  traffic. 


B.2.2  Limiting  Behavior 

We  consider  the  two  cases;  first,  when  B goes  to  infinity,  and 
second,  when  all  arrival  rates  increase  uniformly  toward  infinity. 


Infinite  waiting  room  B ■*  00 

For  the  existence  of  a steady  state,  it  is  necessary  that 
< 1 (i  = 1,  ...,  R) . With  this  condition,  we  expect  that  at  B = 00 
the  system  becomes  equivalent  to  R independent  M|M|l  queues.  From 


Eq.  (B.20)  and  since  < 1, 


R A. 


i1"!  "o  - £ rrV 


B -*•  °°  i=l  Ki 

which  is  the  value  of  g(t)  (Eqs.  (B.16)  and  (B . 18))  at  t = 1,  hence. 


p0  - n (1  - Pi) 

i=l 


R n. 

Thus  from  Eq.  (B.12)  P(n)  = JJ  (1  - p.)p.X 

i=l  1 1 


Moreover,  from  Eq.  (B.23)  we  can  see  that  at  the  limit 
Pr(ni  - V = pi  ">  *Vni  = j)  * (1  ~ Pi^Pj 


The  last  three  equations  prove  the  fact  that  the  system  is  equivalent  to 
R independent  m|m|i  queues. 

Note  also,  that  from  Eq.  (B.19)  and  (B.22) 


limit  G(B)  =0  =>  limit  PB  = 0 


B -*■  00 


B -*■  °» 


Uniform  increase  of  input  rates 

We  now  intend  to  let  all  input  rates  increase  proportionally 
until  eventually  reaching  infinity.  At  this  point  we  show  that  our 
system  becomes  equivalent  to  a closed  network  of  R queues  and  B cus- 
tomers. Let 


Ai  = nAi  i = 1,  . . . , R 


(B. 26) 


where  n is  a positive  real  variable  and  is  a constant.  TTie  service 
rates  (pC)  are  maintained  constant.  Eq.  (B.26)  is  also  equivalent 
to  saying  that  = np°  with  p?  constant. 

From  the  above  definition  and  Eq.  (B.15), 


0 R P K 


where  GU(K)  = £ A. (pT) 
i=l  1 1 


G(K)  = nkG°(K) 


Notice  that  A^  is  invariant;  hence,  G^(K)is  a constant.  As  a result. 


(B.  27) 


and  from  Eq.  (B. 15) 


pn  * £ H G (K)  =>  limit  P«  = 0 


Furthermore,  from  Eq. (B.21) 


(0  n i B 
limit  P = < 

H 00  n ( 1 n = B 


In  other  words,  as  expected,  the  system  is  full  with  Probability  1 
(probability  of  blocking  is  consequently  equal  to  1). 

With  regard  to  the  joint  queue  length  distribution,  Eq.  (B.12) 


becomes. 


P(n)  = 


R 

£ ni 

,i=l  1 


£ nkG°(k)  isl 


R ni 

n>°) 


¥ n e R 


A«) 


limit  P(n)  = < G (B) 


for  n e F.  and  s.t.  E.n.  = B 

b ii 


otherwise 


(B.28) 


With  respect  to  the  marginal  distributions  from  Eq.  (B.24), 

limit  P [n  > j]  = G-^  ~ (p?)j 

n * r G°(B)  ' f 

Furthermore,  the  limiting  throughput  of  type-i  customers  is 


(B  .29) 


limit  (1  - PB)nX?  = --  ^ X° 

n -*• 00  x g°(B)  1 


(B.  30) 


The  above  results  are  typical  of  a closed  network  of  queues  with  R 
service  centers  and  B customers,  such  as  Buzen's  central  server  model 
[BUZE  71],  In  fact,  if  we  consider  the  buffers  as  customers  and  if  we 
let  X^/EX^,  i.e.,  X?/£X?  , be  the  relative  rate  of  arrivals  to  server  i 
(i.e.,  rate  of  allocations  of  freed  buffers  to  server  i),  and  n.  be  the 
number  of  buffers  allocated  to  server  i,  then  the  distribution  of  buf- 
fers is  given  by  the  above  expressions  (Eqs.  (B.28)  and  (B.29)). 
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Moreover,  Eq.  (B.29)  shows  that  there  is  a non-zero  probability 
that  server  i is  idle  ( P^nj  = o]  = 1 - G°(B  - l)p?/G°(B)).  Therefore, 
even  with  infinite  input  rates,  server  i is  not  fully  utilized,  which 
was  not  the  case  with  CP  (see  Eq.  (B.10)). 

The  numerical  example  below  illustrates  the  general  and  limit- 
ing behavior  of  this  system  with  respect  to  n.  In  this  example  we 

assume  that  R = 4,  B = 20,  pj  = 0.1,  p®  = 0.4,  p®  = 0.6,  p®  =0.9 

. , 0 
and  we  let  p.  = qp.  . 

l 

The  utilization  of  server  i is  p|  ■ (1  - PB)p^  = (1  - PB)qp?  • 
The  limiting  value  of  q(l  - PB)  = 1.111;  hence,  n -*■  °°  =>  p'  = o.llll 
p2  = 0.4444,  p^  = 0.6666,  p'  - 0.9999.  Notice  that  server  4 reached 
saturation  whereas  the  others  are  very  far  from  it.  The  average  total 
limiting  utilization  is  p = £ p^  = 0.555  instead  of  1 which  could  be 

obtained  with  CP. 

Furthermore,  Fig.  B.3  shows  the  behavior  of  the  average  number 
of  type-i  customers  in  the  system  (i  = 1,  ....  4)  with  respect  to  n. 
Also  represented,  is  the  average  total  number  in  system,  n.  The  limit- 
ing values  for  the  averages  are  (obtained  at  q = 20) 

n1  -*•  0.125,  n2  + 0.800,  n3  1.99,  n4  -*■  17.02  and  ii  19.94  as  B. 

Notice  that  for  large  q,  most  of  the  buffers  are,  on  the  average,  used 
by  type  4 customers.  Also,  a sharp  increase  of  (from  4.8  to  14) 
occurs  when  q varies  from  0.95  to  1.5.  The  value  of  q = 1.111  would 
correspond  to  the  saturation  of  server  4 (i.e.,  p4  as  1)  if  there  were 
no  limitation  in  buffer  storage,  and  at  that  point  the  queue  size 
becomes  infinite.  This  explains  the  shaip  increase  in  n4,  mentioned 
above. 
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In  summary,  we  conclude  that  with  quite  asymmetrical  utili- 
zations {p?},  CS  tends  to  favor  the  server  with  the  highest  utilization 
even  though  it  has  reached  saturation.  Furthermore,  the  other  servers 
are  left  with  very  little  space  to  share.  These  considerations  motivate 
the  schemes  studied  in  the  rest  of  this  appendix.  Before  we  proceed, 
let  us  apply  the  general  results  obtained  in  this  section  to  the  case 
where  all  p^s  are  equal. 


B.2.3  Special  Case:  Equal  p^'s 

This  section  deals  with  the  case  where  all  the  p^'s  are  equal 
and  we  let  p be  the  common  value.  As  a result,  a simpler  expression  is 
obtained  for  G(K) , and  thus  for  the  other  variables  and  distributions. 

G(K)  is  the  well  known  expression  obtained  for  networks  of 
queues  [KLEI  75], 


Thus,  from  Eq.  (B.15) 


And  from  Eq.  (B.22) 


Also 


hence 


In  the  above  proof,  we  used  the  identity 


Similarly,  the  average  time  in  system  of  the  non-rejected  type-i 


Of  interest  are  the  two  cases  when 


We  frequently  used  Eq.  (B.36)  in  the  derivations  of  the  above  equations. 
Note  that  all  states  n are  of  equal  probability  Pq.  Furthermore,  if 
R = 1 we  find  the  results  obtained  in  Section  B.l  (Eq.  B.ll). 

The  expression  of  PB  may  be  rewritten  as  1/(1  + B/R),  which  is 
exactly  the  same  as  for  a single  m|m|i  queue  with  B/R  buffers  (Eq.  B.ll), 
This  means  that  for  p = 1,  CS  and  CP  lead  to  the  same  probability  of 
blocking.  This  fact  will  be  illustrated  in  the  figures  below. 


The  service  rates  (yC^)  are  assumed  to  be  constant.  The  limits 


po-° 


PB  -*■  1 


n..  -*■  B/R  , n B 


K - »ci 

^ B + R - 1 1 


(B. 39) 


i = 1,  ....  R 


i = 1,  ....  R 


As  noticed  earlier,  infinite  input  rates  do  not  lead  to  full  utilization 
of  the  servers  (except  for  R = 1) , but  only  to  a fraction  B/(B  + R - 1) 
of  the  capacity. 

The  behavior  of  the  probability  of  blocking,  PB,  the  throughput 
normalized  with  respect  to  the  capacity,  Xr/yC^,  and  the  delay  normal- 
ized with  respect  to  the  average  service  time,  yCtj,  are  illustrated 
respectively  in  Figs.  B.4  - B.6. 

This  concludes  the  analysis  of  the  complete  sharing  (CS)  scheme. 
Let  us  now  compare  it  with  the  complete  partitioning  (CP)  scheme. 
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B.2.4.  Comparison  of  CP  and  CS 


I 

i 

I 

i 

: 

j 


: 

! 


We  assume  that  all  p^s  are  equal  (to  p)  and  that  each  server 
"contributed"  BQ  buffers,  i.e.,  ^ = BQ  i = 1,  ....  R (see  Fig.  B.l.a), 
therefore,  B = RBQ. 

With  the  above  conditions  the  behavior  of  CP  (any  queue)  is 
identical  to  CS  with  R = 1. 

Fig.  B.7  illustrates  the  behavior  of  the  probability  of  block- 
ing, PB,  with  respect  to  p and  for  a set  of  values  of  R,  (R  = 1,  . . . , 4) . 
R * 1 corresponds  to  CP;  R = 2,  3,  4 corresponds  to  the  merging  of 
2,  3,  4 single  queues.  Note  that  all  the  curves  meet  at  p = 1 where, 
from  Eq.  (B.38),  PB  = 1/(1  + BQ)  . Note  also  that  for  0 _<  p _<  1,  CS  leads 
to  a smaller  PB,  hence  a better  performance  than  CP.  This  improvement 
is  quite  considerable  for  small  values  of  B^  and  increases  with  R. 
However,  for  p _>  1,  CP  shows  a slightly  better  performance  (smaller  PB) 
than  CS,  namely  for  small  values  of  B^. 

Fig.  B.8  shows  the  respective  channel  utilizations  p (1  - PB) 
(normalized  throughputs  A'/pC).  Note  the  loss  in  limiting  throughput 
(p  -►  °°)  with  CS  for  small  values  of  B^. 

Finally,  Fig.  B.9  shows  the  respective  average  delays.  We  note 
that  the  average  message  delay  increases  as  more  buffers  are  provided, 
i.e.,  as  R increases. 

The  slightly  better  performance  of  CP  for  p > 1 intuitively  in- 
dicates that  some  buffers  should  be  permanently  allocated  to  each  server. 
This  idea  is  incorporated  in  Scheme  4,  SMA.  Moreover,  we  observed 
earlier  that  very  unbalanced  input  rates  lead  to  uneven  (on  the  average) 
usage  of  the  storage  space.  This  remark  motivates  the  next  scheme,  SMXQ. 


* 


' -J 

‘ I 


335 


.2 

> : Utilization. 


B.  3 


Sharing  with  Maximum  Queue  Lengths  (SMXQ) 

Like  CS,  SMXQ  allows  the  sharing  of  a pool  of  B buffers  with  a 


further  constraint  imposed  on  the  number  of  buffers  to  be  allocated  to 
any  server,  and  at  any  time.  Let  b^  be  the  maximum  number  of  buffers 
that  can  be  used  by  type-i  customers;  the  set  of  feasible  states  becomes 

, R . 

Fc  = { 2 | 0 1 JL  ni  1 B » 0 1 ni  1 bi  i = 1,  ....  R} 

Taking  advantage  of  a previous  remark,  we  directly  write  the  set  of 
local  balance  equations  which  describe  the  behavior  of  the  system  of 
queues  in  steady  state.  We  consider  below  the  possible  transitions  for 
type-i  customers, 

1. 

X.P  (n  j,  • • • » n^  - 1,  ...,  n + P^P  ^1  ’ • • • » + if 


= + ^JPCnj, 


n^  f • • • f n^) 
for  all  n s.t. 


0 < n.  < b. 
l l 


I\ 

0 < T n.  + 1 < 

- h J 


0 < n . < b . 
- 3 ~ 1 


j * i 


2. 


AiP(n1,  . 


n^  - 1,  ...,  n^)  = •••f  n^>  ••«»  n^) 


for  all  n s.t.  / n.  = 1 and  / 0 < £n.  < B 

i I - 3 - 


or 

ni  - bi 


0 < n.  < b. 
“ 3 ~ J 


and 


n.  > 1 and  ( I .n . = B 
i “ I j j 


0 < n.  j?  bj 


Again,  it  is  easy  to  check  that  the  joint  probability  distribution 
satisfies  Eqs.  (B.12)  and  (B.13)  where  is  replaced  by  Fc< 

The  evaluation  of  is  much  more  complicated  here,  because 
of  the  added  constraint  on  n^.  In  what  follows,  we  again  consider  the 
two  cases  of  different  and  equal  p^'s. 


B.3.1  General  Case 

In  this  section  the  p^'s  are  not  necessarily  equal.  We  first 
evaluate  P^.  From  the  above  considerations 


p0  = £ Q« 

(B.40) 

K=0 

where 

_ "l  nR 

QW  - V Pj1...  PRR 

(B. 41) 

R 


0<n . <b . 

— l—i 

Note  that  the  difference  between  Q(K)  and  G(K) , Eq.  (B.14),  comes  from 
the  added  constraint,  n.  < b.. 

In  order  to  find  Q(K) , we  use  a method  similar  to  the  generat- 
ing function  approach.  Let  f(t)  be  defined  as 


2.2 


f(t)  = II  ( 1 + p.t  + pft 

i=l  1 1 


b.  b. 

* p/t x) 


(B. 42) 


By  expanding  the  above  product,  we  recognize  that  Q(K)  is  the  factor  of 
K . 

t , i.e. , 


f(t) 


Q(0)  + Q(l) t + ...  + Q(K)tK  + 


R 

Note  that  Q(0)  * 1 and  that  the  highest  degree  is  equal  to  £ b.. 

i=l  1 
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The  function  f(t)  may  be  rewritten  as 


g(t)  = f(t)h(t) 


The  above  relation  will  allow  us  to  express  G in  terms  of  Q and  C 
The  term  CL  results  from  the  partial  fraction  expansion  of  h(t). 
Assuming  that  all  p.'s  are  different,  we  find 


where 


Finally 


k(l+b.) 


5 


TTien  equating  the  terms  of  equal  degrees  in  t,  we  arrive  at  a relation 
between  G,  Q and  C^'s.  rhis  relation  is  quite  complicated  and  requires 
the  ordering  of  the  b^'s  and  their  multiples.  However,  if  we  restrict 
our  considerations  either  to  the  case  where  bi  _>  , or  when  bi  = b 

for  all  i,  then  we  obtain  the  simple  relations  below. 

i.  We  assume  that  each  queue  is  allowed  to  occupy  more  than 
half  of  the  entire  space,  i.e., 

b±  > B/2  ¥ i = 1,  ...,  R (B. 49) 

then  2b^  + 2 > B and  b^  + b^  + 2 > B.  Therefore,  in  order  to  find 

Q(K),  K = 0,  1,  ...,  B,  we  only  need  to  consider  the  terms  in  h(t) , 

of  degree  0 or  1 + b..  Those  terms,  obtained  directly  from  Eq.  (B.44) 

(i.e.,  without  the  assumption  of  different  Pi's),  are 

R 1+b. 

1 + L (P^t)  1 

i=l  1 

Substituting  the  above  expression  into  Eq.  (B.48),  we  arrive  at 

R 1+b. 

G(K)  = Q(K)  + £ p.  XQ(K  - b.  - 1)  (B.50) 

i=l  1 1 

i s.t.  b.  < K 

l 


Then, 


(l+b)K 


Substituting  the  above  into  Eq.  (B.48),  we  arrive  at 


i)(b  + 1)  + k)L(i) 


where  0 < k < b 


Note  that  G(0)  = Q(0)  = L(0)  = 1,  and  that  Eq.  (B.54)  allows  the 


sequential  computation  of  the  sequence  Q(K)  for  K varying  from  1 to  B 
Note  that  if  b ^ B/2,  then  Eq.  (B.54)  becomes 


Q(k)L(l) 


thus  the  combination  of  the 


last  three  equations  gives  Eq.  (B.50)  with  b.  = b 


In  what  follows,  we  restrict  our  considerations  to  the  case 


where  b.  B/2,  unless  specified  otherwise.  As  a result,  and  from 


Note  that  if  b.  * B for  all  i 


and  the  above 


equation  reduces  to  Eq.  (B.15). 

1 a-1  . 

By  convention  we  set  £ * 0 ¥ a integer 


Similar  to  Section  B.2,  we  now  assume  that  all  the  p^s  are 
different  (Pi  + P-  ¥ i t j);  then  using  Eqs.  (B.19)  and  (B.20),  we 
arrive  at 


1 - pB+1 


« *■  ~ k 1+b.  R l - p. 

-0 

Osb . <B 

1 (B. 56) 

Eq.  (B. 12)  with  Fb  replaced  by  F£,  and  Eq.  (B.55)  or  (B.56)  completely 
characterize  the  queueing  system  under  SMXQ  and  the  condition  of  Eq. 
(B.49).  Similarly  to  Section  B.2,  we  proceed  with  the  derivation  of 
distributions  and  average  quantities  of  interest. 

Distribution  of  the  total  number  in  system:  Probability  of  Blocking 
Let  n be  the  total  number  in  system;  then,  for  0 n <_  B, 

Pn  ■ Wi  ' "1  - Po«"> ' 

r R 1+b.  -« 

Pn  = Po[G(n)  ' £ pi  1(5(n  - bi  ' (B.S7) 

b.<n 

l 

Let  us  now  derive  the  probability  cf  blocking  of  type-i  customers,  PB. . 
Recall  that  type-i  customers  are  hocked  if  upon  arrival  the  entire 
space  is  full,  or  if  the  number  ot  type-i  customers  is  equal  to  b.. 
Since  arrivals  are  Poisson,  then 


1 - p. 


B-b. 


PB 


i * prPjnj  * B or  ni  = b±] 


(B. 58) 


which  is  also 


PB 


i = Pr^Zjnj  = B1  + prlni  * bi  3,1(1  2jnj  < B] 
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Moreover 


Because  of  Eq.  (B.49) 


Hence 


where 


Finally 


where  PR  is  given  by  Eq.  (B.57). 
Note  that  G. (K)  is  similar  to  G(K) 


except  that  we  deleted  p.  (i.e 


We  now  proceed  with  the  derivation  of  the  marginal  distribution 
and  average  number  and  delay  of  type-i  customers. 


Marginal  distribution  and  averages 


The  summation  above  is  similar  to  that  of  P'1  except  that  B is  now  B - 
and  the  component  ni  is  deleted.  Also  note  that  _>  k it  j / i, 

hence  the  above  summation  is  given  by  Eq.  (B.55)  where  B is  replaced  by 
B - k and  G(K)  by  G^K)  (as  defined  in  Eq.  (B.59)).  Thus, 


The  above  summations  can  be  further  reduced  to  expressions  similar  to 
the  one  in  Eq.  (B.56). 

As  for  the  average  number  of  type-i  customers. 


With  respect  to  the  throughput  X'  and  the  delay  t , the  expressions 
XU  (1  - PBi)Ai  and  ^ = n±/XJ  still  hold  true. 

This  terminates  the  characterization  of  the  system  as  operated 
with  SMXQ  and  with  the  assumption  of  b±  > B/2  and  different  pj's.  Next 
we  study  the  case  of  equal  pi's;  we  leave  numerical  applications  to 
Section  B.6. 


Similar  to  Section  B.2.3 


given  by  Eq.  (B. 31) . Also,  we  assume  that  all  the  b.' 

1 

and  that  b > B/2.  Therefore,  from  Eq.  (B.55-)  Pi1  been 


As  for  the  distribution  of  the  total  number  of  customers,  from 


Eq.  (B.57)  we  obtain  (for  n < B) 


P = 
n 


><,("  r - ; y . 

[C ; v)-*(n  - r : r 2)] ’n 


0 < n < b 


b < n < B 


(B.64) 


In  order  to  find  the  probability  of  overflow  PB  (the  same  for  any  type 
of  customers),  we  need  to  evaluate  G. (K)  (Eq.  (B.59));  similarly  to 
G(K) , we  have 


■ (K ; ? ; y 


(B. 65) 


Substituting  Eq.  (B.64)  for  n = B,  and  Eq.  (B.65)  into  Eq.  (B.60), 
we  arrive  at 

"I-™- vB[(B;v)-*(’-b*:r1)] 

* v £ I * - * )p 

With  regard  to  the  marginal  distribution  of  any  type  of  customer,  say 
i,  from  Eqs.  (B.61)  and  (B.65),  we  find 


(B. 66) 


Pr[ni=k]  = P0P 


B-k 

4 

K 


V*  / * + R - 2\  K fX}  ..  b+1  B"b-k"1/  K + R - 2\ 

£(  r -(R‘1)P  £ ( r 


for  k <_  b 
otherwise 


(B. 67) 


The  rest  of  the  expressions,  n^.  A',  t..,  follow  in  the  same  way  as 
before.  Of  interest  is  the  case  where  p = 1 and  p -*•  <» 

p = 1 : 

Using  the  identity  in  Eq.  (B.36),  we  arrive  at 
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Note  that  if  R « 1.  then  PB  * Pq  * l/(b  + 1),  which  is  exactly  the  prob 
ability  of  blocking  (and  P.)  for  a single  m|m|i  queue  with  b buffers; 


because  of  the  constraint,  n.  < b 


unutilized 


Note  that  for  the  two  limiting  cases  where  b = B,  i.e.,  CS  and  b = B/2 


i.e.,  CP,  we  have  PB  = 2/ (B  + 2);  which  checks  with  the  previous  results 
(Sections  B.l  and  B.2).  Moreover,  a question  arises  as  to  choice  of  b. 
This  question  and  numerical  examples  are  presented  in  Section  B.6. 

Let  us  now  consider  the  case  when  p goes  to  infinity. 


Of  interest  is  the  limiting  utilization.  Let  p'  be  the 
utilization  of  any  server;  then  p"  = (1  - PB)p.  Before  we  proceed 


note  that  P.  -*•  0,  PB  -*•  1 


Note  that  if  R • 1 then  p'  - 1 (provided  that  B > 1);  whereas,  for 


R = 2 (B  ^ 1) , two  cases  are  possible 


Both  results  are  to  be  expected,  b < B implies  that  no  one  of  the  two 
types  can  utilize  the  whole  space,  and  hence,  at  p = °°  the  two  types 
are  always  present,  b = B implies  CS;  hence  we  are  back  to  the  results 


II 

| 

h 

it 

f! 


of  Eq.  (B.  39)  for  Xf/yCL  and  R = 2. 

B*4  Sharing  with  Minimum  Allocations  (SMA) 

Similar  to  CS,  SMA  allows  the  sharing  of  a pool  of  B buffers, 
and  in  addition,  a i buffers  are  permanently  allocated  to  type-i  cus- 
tomers, i = 1,  ...,  R (see  Fig.  B.l).  As  a result,  the  set  of  feasible 
states  becomes 

f R 

Fd  = 1 5 I 2 SUP  {0,  n.  - a.}  < B , 0 < n.  < B + a i = 1,  ...,  r! 

' i=l  A A 1 1 / 

Here  also,  we  can  write  the  local  balance  equations  and  verify  that  the 
product  form  solution  satisfies  those  equations.  Following  the  same 
steps  as  earlier,  we  first  consider  the  general  case  of  different  Pi's. 

B.4.1  General  Case 

In  order  to  evaluate  PQ,  we  partition  the  set  Fd  into  disjoint 
subsets  which  lead  to  known  summations.  Let  &l  be  the  set  of  customer 
types, 

&t  = {l,  2,  ...,  R} 
and  let  be  the  set  of  all  subsets  of  £*?, 


•as mwt 


ft; 

j$Y  ■ 


I 


The  set  SC  contains  2 elements;  among  them  are  the  set  itself  and 


the  empty  set.  We  then  associate  with  each  subset,  X , a subset  of  F . 

m d 


defined  as 
m 


e F . n. 

d ' i 


> a-  i e X 
— i m 


< otherwise 


i C a l 


Equivalently, 


o J ieX 

S = { n m 

m / ~ 


£ (n.  - a ) < b,  a.  < n.  < B + a.  for 

ic-Y  11  1 — 1 — 1 


i e X 


n.  < a for  i i x I 
i x T m ’ 


Obviously, 


S O S = 4>  m t n and  F,  = u s 
m n d Y,  m 

m=l 


Therefore, 


R n. . 2 


l ^ ^ n4v  / R n.v 

o1  - £ ( n Pi1  - £ £ ( n p,1 

neF,  'i=l  / m=l  neS  'i=l  ’ 


(B. 71) 


Let  H be  the  summation  over  all  states  in  S , then 
ra  m 


Hm=  £ ( n p"1)-  n ( £ p"1) 

neS  ' i=l  ' i$X  ' Osn.<a.  / 


m ii 


£ n Pi] 

Os  E (n.-a.)sB  ieX 


aisnisB+ai 


Hence, 


a. 

H"  14* „ 1 ' °1  ) 0<  & 0l 

" “ icX  1 " 

m 


Osn . SB 

l 


Let  us  define  the  generating  function  C , such  that 

m 


‘ i 

I 


I 


(B.  72) 


c„m  * £ /n  PiM  0.72) 

Z n . =K  \ieX  1 / 

ieX  1 ' m 

m 

0<n . <K 
i 

Cm(K)  is  similar  to  G(K)  given  in  Eq.  (B.14),  thus  it  can  be  computed 
in  the  same  way.  Finally,  let  a = (a^  ....  aR) . Then 


P.o‘  « - P0I  ■ £ H 

m=l 

a. 

A 1 - P/ 


a.  B 


(B. 73) 


HJs- - hb  - n ^ n Pl‘  E v* 


Note  that  if  a.  = 0 for  all  i,  then  St  = , Cm(K)  = G(K) , and  the 

above  equation  reduces  to  the  description  of  CS.  Also,  the  summation  of 

C (K)  is  set  to  1 if  X = d>. 
m m T 

If  we  now  assume  that  all  p^s  are  different,  then  from 
Eqs . (B.18)  and  (B.19), 


C (K)  = £ A.  pK 
m . - i,m  l 


(B . 74) 


where 


A.  = IT  T- 

1*m  keX  1 


pk/pi 


(B. 75) 


Using  a similar  summation  as  in  Eq.  (B.20),  we  arrive  at 


h_  = n 

m . i „ 


a. 

1 - p/ 


B+l 
1 " P; 


(B. 76) 
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Eq.  (B. 12)  with  Ffe  replaced  by  Fd,  and  Eq.  (B.73)  or  (B.76)  completely 
characterize  our  system.  We  now  proceed  with  the  derivation  of  dis- 
tributions and  averages  of  the  variables  of  interest. 

Distribution  of  total  number  of  customers  in  shared  area;  Probability 
of  Blocking 


Let  ns  be  the  total  number  of  customers  in  the  shared  area. 


i.e. , 


R 


n = £ sup  {o,  n - a.} 
i=l  11 


(B • 77) 


Then  the  distribution  of  ng  is,  for  k <_  B,  equal  to  the  sum  of  proba- 
bilities of  states  n which  satisfy  Eq.  (B.77)  for  ng  = k.  In  order 
the  evaluate  that  summation,  we  use  the  same  methodology  as  for  the 

determination  of  P_.  Let  F,(k)  C F,  and  S (k)  G S be  defined  as 
u d d m m 


Fd00 


f R 

<n  | Y.  SUP  l°»  ~ a.}  = k,  0 < n.  < B + a.  I 

i=l  11  “ 1 “ 1 f 


< n 

a.  < n.  < a.  + k 

l—i—i 

i e X 

m 

L (ni  - ai)  = k) 
ieX  1 1 \ 

( r 

n.  < a. 
i i 

i i X 
r m 

" 

It  is  obvious  that  F^(k)  = ^ Sm(k) , and  consequently 


V*s  - ki  - p0  i z (np-M 

m=l  neS  (k)  ' i=l  ' 


m=l  neS  (k) 
~ m 


(B. 78) 


Let  h (k)  be  the  summation  over  all  states  in  S (k) , then  similar  to 
1,1  m 

the  above 

2R 

- k)  * p0  £ vk> 

m=l 

a. 

h"(k)  • i?x  (&  P“i)C«(k) 


(B. 79) 


PB  can  be  computed  similarly  to  P [n  = Bl  with  a restriction  on  the 
r r s 

choice  of  the  subsets  X , which  must  contain  r,  i.e., 

PBr  = P0  £ hm(B)  (B.82) 

m 

m J reX 
1 m 

R- 1 

There  are  2 such  sets  which  can  be  obtained  as  follows.  Let 

= uft  - {r}  and  = {x^  n = 1,  . ..,  2P  be  the  set  of  subsets 
of  then  Xn  = X'  y{r}  i-s  such  a subset  of  which  contains  r. 


Marginal  distribution  and  average  number  and  time  in  system 


We  now  proceed  with  the  derivation  of  the  marginal  distribution 

of  the  number  of  type-i  customers;  more  precisely,  we  intend  to  find 

prtnr  ±.  j]*  We  consider  the  two  cases  which  depend  on  whether  j is 

less  than  a or  not. 
r 

i.  0 < i < a 
— J r 

• n,  n -j  nn 

prtnr-  j]  = pQpJr  L pj  •••Prr  •••prR 


^ ; 
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< 


p: 


Let 


i i r 


a.  = 

1 


(B. 83) 


a - l 
r J 


be  the  new  minimum  allocations,  and  ?'  be  such  that 

d 

l R 

Fa  = » n I 23  SUP  {o,  n.  - a'}  B,  0 < n < B + a'  i 

u ' ~ i=l  11  “ i — 1 J 


Therefore 


n,  n 


Pr[nr  > j]  ■=  P0Pj  £ P11"‘P/"‘PRR 


neF" 

- a 

The  summation  above  is  similar  to  the  one  of  P"1,  Eq.  (B.71),  where 
Fd  is  replaced  by  F^.  Let  us  define  the  vector  a'  = ( a a',  ....  a'); 
then,  replacing  a by  a'  in  Eq.  (B.73),  we  find 

. 2R 

Prtnr-  jl  = P0PrP0  [*’  B]  = P0pr  £ B1  <B-84) 

m=l 


V j < a. 


ii.  ar  < j £ B 


j-a  r-,  n1  n^- j+a 

Prt"riJ]  - p0pr  2-p! 

neFd 


r V*  "l  ,,r"J'"aT  nR 

2mJ  D-  • • *D  •••0 

r 


nr^ 

To  evaluate  the  above  summation,  let  us  define  the  set 

R 


n 

r 

i=l 

sup  {0,  n.  - ai}  _<  B - j ♦ arJ 

( 

0<n.  <B-j+a  + a.  ) 
— l — r l 

Then,  replacing  n^  - j + a^  by  nr,  in  the  above  equation,  we  arrive 
at 
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j-a  y*  n1  n nR 

P [n  > jl  = Prtp  r ^ p,  ‘‘‘p  r**,p_ 
rl  r— (Tr  neF"  r Mr 

~ d 
n £a 
r r 

The  set  F''  is  similar  to  except  for  the  condition  n^  j>  ar,  and  the 
change  from  B to  B - j + a^;  then  similar  to  the  derivation  of  PBr> 

j -a 

prtnr  ± j]  = p0Pr  r E HmCa,  B - j + ar)  for  &r  < j < B 

m 

m | reXm  (B.85) 

Eqs.  (B.84)  and  (B.85)  give  the  marginal  distribution  of  the  number  of 

type-r  customers.  As  a result,  we  can  derive  n , X'  and  t . 

r r r 

Let  us  now  apply  the  above  results  for  the  special  case  of 
uniform  utilization  and  allocation. 


B.4.2  Special  case:  p^  = p , a^  = a 

The  assumption  of  equal  p^'s  and  a^'s,  leads  to  much  simpler 

expressions  of  the  variables  above.  First,  if  p is  the  size  of  the 

subset  X (p  = |x  I),  then  from  Eq.  (B.76) 
m m 1 


,«■(*;?;  V 


Note  that  if  p = 0 then 


c 00  = 

m 


0 K > 0 


1 K = 0 


(B. 86) 


(B. 87) 


Also,  from  Eq.  (B.77) 


R-p  Q 


(B. 88) 
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Note  that  C (K)  and  H (k)  depend  only  on  the  size  p of  the  set  X . The 
mm  m 

number  of  sets  of  size  p is  equal  to  (R)  , thus  from  Eqs.  (B.73)  and 


(B. 88) , 


tf-id (Hf  f" pa  |0  r ; M 


(B.  89) 


Similarly,  we  derive  the  expression  for  PBr  (Eq.  (B.82).  Recall  that 
we  only  account  for  sets  Xm  which  contain  r,  hence  p >_  1 . 

pBr-0r(pR:|)(Kf)R'V(B;-y  ».«) 

r = 1,  . . . , R . 

Note  that  in  the  above  expressions,  a was  assumed  to  be  greater  than 

zero;  if  a = 0,  then  all  subsets  X_  are  empty  except  one:  X = a?  . 

m m 

whose  size  is  equal  to  R.  Moreover,  with  a = 0,  SMA  reduces  to  CS 
and  Eqs.  (B.89)  and  (B.90)  for  a = 0 check  with  Eqs.  (B.32)  and  (B.33). 
Now,  if  B = 0 then  SMA  reduces  to  CP,  and  in  fact,  Eqs.  (B.89)  and 


(B.90)  become 


P a 

-ferp 


Let  us  now  derive  the  marginal  distribution  of  the  number  of  type-r 
customers.  Eq.  (B.84)  provides  _>  j]  for  j < a^;  the  terms 

H [a'',  B]  can  be  evaluated  as  in  (B.89)  except  that  a'  = a - i. 

In  -w  TT 

Therefore,  we  must  distinguish  the  sets  X which  contain  r from  those 

m 

which  do  not.  If  p = | Xm|  , then  (^  " j ) such  sets  contain  r and 
(R  ~ *)do  not.  As  a consequence 
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•[”r ■ po  i ( p : J )( H£)  V h(* ; i ; y 


R-p-l 


(B.91) 


For  the  case  where  j > ar,  then  from  Eq.  (B.85)  and  using  the  same 


procedure  as  above,  we  find 


p^-jl -p.£(; |a(Kp-i>K 

(B.92) 

With  respect  to  n^,  X^,  t^  , they  follow  from  the  above  considerations. 
Of  further  interest  is  the  limiting  behavior  when  p goes  to 

infinity. 


m 


We  expect,  with  the  minimum  allocation  of  a buffers  per 
server,  to  obtain  a full  utilization  of  all  servers.  The  terms  of 
highest  degree  in  P^1,  PBr  and  p(l  - PB^  are 


(B;.V)pB*Ra 

[(b*/;2)*(b;b22)] 


for  P"1  and  PB 
0 r 


for  p(l  - PBr) 


Therefore,  p ~ «=>  P -*•  0,  PB  -*■  1,  p(l  - PB  ) -►  1. 

u y r 

This  concludes  the  analysis  of  CSMA,  the  numerical  applications 
and  questions  related  to  the  choice  of  a are  treated  in  Section  B.6. 
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Sharing  with  Maximum  Queue  Length  and  Minimum 


Allocations 


In  addition  to  SMA,  SMQMA  (or  scheme  e)  imposes  a constraint 
on  the  number  of  buffers  from  the  shared  pool  to  be  allocated  to  any 
server,  and  at  any  time  (see  Fig.  B.l).  Let  b.  be  that  constraint  with 


Equivalently 


Here  also,  we  can  check  that  the  product  form  solution  satisfies  the 
local  balance  equations.  Thus  we  may  proceed  as  earlier  with  the 
evaluation  of  and  subsequently  determine  the  distributions  and 
quantities  of  interest.  First  we  consider  the  general  case  of  differ- 


General  case 


The  same  procedure  as  in  Section  B.4  can  be  utilized  to 
partition  the  set  F into  disjoint  subsets  which  then  lead  to  known 


summations.  Those  subsets  are 


Consequently 


i 


: 


with 


1 - P 


a. 

1 


hc  = / n , — -i-)(  n 

" 1 - pi  M icx_ 


where  Q(K)  is  as  defined  in  Eq.  (B.41) 


(B.94) 


As  a consequence,  the  computation  of  P”^  follows  as  in 
Sections  B.3  (for  Q(K))  and  B.4.  This  remark  holds  true  for  the  compu- 
tation of  the  summations  which  may  appear  in  the  analysis  of  this  v 
scheme.  As  a result,  we  need  not  carry  the  study  of  this  scheme  any 
further. 


B.6  Further  Numerical  Results  and  Comparisons 

In  this  section  we  intend  to  compare  our  first  four  sharing 
schemes:  CP,  CS,  SMXQ,  SMA,  under  the  assumption  of  equal  p^'s.  Before 
we  proceed,  let  us  recall  that  if  B is  the  total  number  of  buffers 
(B  = RBq)  and  b is  the  maximum  queue  size  (for  any  queue)  when  using 
an  SMXQ  scheme,  then 

1.  if  b = B SMXQ  is  equivalent  to  CS 

2.  if  b = Bq  SMXQ  is  equivalent  to  CP 

3.  if  R = 2 then  SMXQ  is  equivalent  to  SMA  with  a minimum 
allocation  per  queue  equal  to  B - b. 

As  a result  of  the  above,  the  study  of  SMXQ  with  R = 2 and  a variable 
b will  allow  us  to  cover  the  four  sharing  schemes  to  be  considered  here. 

In  the  numerical  example  below  we  assume  that  R = 2,  B = 6 
and  that  b satisfies  B/2  < b < B (see  Eq.(B.49),  i.e.,  b = 3, 4, 5, 6. 

From  our  previous  considerations  we  know  that  b = 3 leads  to  CP,  b » 4 

and  b * S lead  to  non-degenerate  SMXQ,  SMA,  and  b = 6 leads  to  CS. 


■i 


3S9 




Figs.  B.10,  B.ll  and  B.12  show  respectively  the  probability  of 
blocking  PB,  the  channel  utilization  p(l  - PB) , the  normalized  average 
message  delay  yCT,  obtained  with  the  four  schemes.  With  respect  to 
blocking  and  utilization,  the  optimal  b (i.e.,  the  optimal  scheme)  is  a 
function  of  p.  We  note  that  for  small  values  of  p,  b = 6 (i.e.,  CS)  is 
optimal;  as  p increases  b = 5,  then  b = 4 (i.e.,  (SMXQ,  SMA)  become 
optimal,  and  finally,  for  a larger  p,  b = 3 (i.e.,  CP)  becomes  optimal. 
With  respect  to  the  average  delay,  it  is  an  increasing  function  of  b. 
There  is,  consequently,  a tradeoff  between  the  probability  of  blocking 
and  the  system  delay.  Therefore,  the  selection  of  a particular  scheme 
should  account  for  these  two  variables  as  well  as  the  load  on  the 
system. 

B. 7 Conclusion 

In  this  study,  we  considered  various  schemes  for  sharing  a 
pool  of  buffers  among  a set  of  coaraunication  channels  in  a communication 
network  environment.  Five  sharing  schemes  are  examined  and  the  results 
of  the  analysis  are  presented  and  displayed  in  a fashion  which  permits 
one  to  establish  the  tradeoffs  among  blocking  probability,  utilization, 
throughput  and  delay. 

We  found  that  no  one  scheme  is  always  optimal;  one  should 
select  a scheme  to  fit  the  particular  operational  environment.  This 
study  shows  that,  in  general,  sharing  with  some  restrictions  on  the 
contention  of  space  is  certainly  more  advantageous  than  no-sharing, 
especially  when  little  storage  is  available. 


0 0.4  0.8  1.2  1.6  2.0 

P 

Figure  B.11.  Comparison  of  the  Four  Schemes  : Utilization. 
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APPENDIX  C 


PROPOSITION  6.1  AND  ITS  PROOF 


APPLICATION  TO  SOME  SPECIAL  CASES 


Optimal  Clustering  Structure 


Proposition  6.1 


Given  m,  the  number  of  levels  in  the  hierarchy,  and  assuming 
that  cu  > 1 for  all  i = 1,  ....  m,  then  the  solution  of  the  optimization 
problem  6.11,  i.e.,  the  optimal  clustering  structure,  is  such  that: 

(a)  All  clusters  at  the  same  level,  k = 1,  ....  m,  are  composed 
of  an  equal  number  of  lower  level  clusters,  i.e.,  all  nodes  at  the  same 


level  in  the  tree  representation  are  of  equal  degree.  The  optimal  degree 
vector  reduces  to  an  m-dimensional  vector,  n = (n. , n_,  ...,  n ),  whose 


components  are  the  solution  of  the  following  set  of  difference  equations 


where  by  convention 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  luACAU  Of  STANCANOA  - 1»*  A - A 


w- 


and  is  the  solution  of 


{ D2  = 1 

k-2 

( Dk  = “k-iDk-i  * n,  fai  - ■>  k i 3 

v i=l 


(6.13) 


Also,  is  the  solution  of 


a. 


B2  = N 


EE) 


k "ak-lDk-l 


°kJ  -Vi 


r k-2 

II  (a.  - 1) 
i=l 

J 


k-2 
T n (a.  -1) 


(6.14) 


i=l 


B,.  , 1°^  - 1 D; 


r^ii 

k-J 


k-l 


k > 3 


(b)  With  this  optimum  solution,  the  minimum  computational 


cost  is: 


G(m,  a,  8)  = B 


m+1 


(6.15) 


During  the  course  of  the  proof,  we  will  first  fix  a set  of 
variables  and  carry  the  optimization  step  over  the  subset  of  non-fixed 
variables.  Then  we  will  replace  the  optimal  values  of  the  non-fixed 
variables  in  the  objective  function  and  repeat  the  procedure  until  we 
exhaust  all  of  the  variables.  Note  that  the  optimal  values  of  the 
non-fixed  variables  will  be  expressed  in  terms  of  the  fixed  variables. 
Step-by-step,  the  above  strategy  will  proceed  as  follows: 

1.  k •*-  1 

2.  Fix  all  variables  n..(im,  i ) V j = k+1,  ...,  m 

Solve  the  optimization  problem  with  respect  to  the 
variables  nk(im,  ....  ifc+1) 
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3.  Replace  the  nfc(  )'s  by  their  optimal  values  in  the 
objective  function. 

4.  If  k = m Stop 

Else  k *■  k + 1 Go  to  Step  2 

Note  that  when  k » m,  there  are  no  fixed  variables,  and  the 
optimization . is  carried  out  with  respect  to  n^. 

To  introduce  some  of  the  ideas  incorporated  in  the  general 
proof,  we  shall  first  solve  for  m * 2. 


Optimality  for  m 3 2 

The  optimization  problem  becomes: 


“l 


min:  G(2,  n,  a,  6)  = £ [n1(i2)]  + [82n2] 


V1 


over:  n 


constraint:  £ n1^2^  = N • n >_  0 


V1 


Claim  C.l 

According  to  Proposition  6.1,  the  solution  of  the  above  problem 
must  be  such  that: 


nl(i2)  ni  * „2  ¥ i2  = 1,  ....  n2 


' a - i a' 

n,  * — — N 

2 a2 

n fO  \ *• 


1 

TOj+c^-l 
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min  G(2,  n,  a,  6)  = (Oj  + <*2  - 1) 


1 


- 

a*2 

a2 

I 

H 

1 

rH 

8 

Ctj 

m 

[ J 

Proof: 

Following  the  strategy  outlined  above,  the  proof  will  proceed 
in  two  steps: 

Step  1 

n2  is  fixed.  Solve  the  problem  with  respect  to  n^(i2)  * 

*2  " *••»  n2» 

The  objective  function,  being  a sum  of  power  functions 
“l 

tn1(i2)l  » which  are  convex  in  the  region  {n1(i2)  > 0,  for  all  i's}, 

is  a convex  function  in  that  region  (recall  oij  >1).  Taking  the 
Langrangian: 

1^2  n 

LfV  - ,Ei  fnjCV]  1 - X[.Lj  ni(i2)  - nJ 

where  n1  = {nj  Ci2) } and  \ > 0. 

Notice  that  we  discarded  the  constant  term  (02n2)  2 and  the 
positivity  constraint,  which  we  will  check  a posteriori. 

The  Langrangian  is  also  a convex  function  (for  njCi2)  > 0) , 
and  at  optimality  iij  must  be  such  that: 

■sr^ip-  ’ 0 * 42  ■ 1 "2 
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p,,u 1 l,JW 


1 1 " ■ " 


Hence, 


“l"1  r -.“l 

Ojfnjfij]  - 1 ■ 0 ->  nj(i2)  - [ij 


1 

a,-l 


^ ^2  = ^ » • • • » 1*2 

which  means  that  all  ^(i^'s  are  equal;  therefore,  from  the 
constraint : 

N 

nl^x2^  = nT  ¥ x2  = *»  *•*»  n2  (notice  n^i^  > 0) 


size 


Step  2 


n2  now  a variable,  and  the  problem  becomes: 

“l  1_ai 

rain  G = N (n2)  + (B^)  2 


over:  n2  _>  0 


Since  Oj,  a2  are  strictly  greater  than  one,  the  objective  is 

dG 
dn 


a convex 


function  in  the  region  (n2  > 0).  Then,  if  n2  solution  of  ^ = o is 


such  that  n2  > 0,  it  must  be  the  optimal  solution. 

“i  -a,  a.  a_-l 


f .0 

*2 


1 “i  a --i 

->  (1  - otjKN)  l(n2)  + a2C62)  2(n2)  2 = o 


Hence 


Notice  n.  > 0 


By  replacing  n_  in  the  objective  function 
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1-a  /a  + a - 1\  a 

G • <"2>  (-■  a"-'  - )W 


1-a, 


a. 


a - 1 

(N) 


a,+a„-l 


a- 


La2(62) 


1 2 a + a_  - 1 a. 

- (N) 


a„ 


which,  after  grouping  the  terms  in  at  the  denominator,  results  in 
the  expression  proposed  by  Claim  C.l. 


General  Case 


In  accordance  with  the  step-by-step  strategy,  we  will  first 
prove  a lemma  that  provides  the  solution  for  the  optimization  problem 
obtained  after  k steps. 


Lemma  C. 1 


The  optimal  solution  of  the  optimization  problem  obtained 
after  fixing  the  variables  n^.(im,  ...,  ij+1)  for  j * k ♦ 1,  ...,  m 
is  such  that 


n^(im»  • • • » ^ (lmi  •••*  Ij) i 1 r 1,  •••»  k 


4+1 


m 


nl  = 


4-1  1 
n a. 


r 4-1 

A(ai  " B r nm  n4+2{*)  1 “i 

“ST  °4  [i?i”i^-l  njl+l(im’  ‘ i)l+2)] 

VS4) 


m 4+2 


4+1 


nk+2(*i 


C m k+^v  ' 

n - N[n  n ...  „ £ — £ n 

^ L i *1  i,..-,=l 


¥ 4 = 2 , 3,  . . . , k 
1 


m 


k+2 


k+l(V  1k+2)J 


(C.l) 
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and  the  minimum  G is  expressed  by: 


?•  !>  * •k.jE  •••  £ Vi(i. wl*  011 

L 1m  J 


iSi'”1'11 


X ***  X f8k+ink+l(la»  •••»  iW2)J 


i -1  i 


(C.  2) 


Proof: 


The  proof  will  proceed  by  induction.  Let  us  prove  that  Lemma 


C.l  is  true  for  k = 1. 

(i)  JLJLJL'  AH  variables,  n^f...)  j ■ 2,  ...»  m , are  fixed. 
Thus  the  optimization  problem  (6.11)  reduces  to: 


min  G 


i * .23  ***  23  ^ni^m*  •••»  ij)i 

1=1  i„=l  m * 


s.t. : size  constraint 


Since  Oj  > 1,  the  above  objective  is  convex  in  the  region  nj's  > 0. 
Taking  the  Lagrangian,  we  get 

Lf®!*  x)  s 6j  - X ( J3  •••  J3  njCi^j,  ...,  i2)  - N ) 

1m  x2 

In  the  region,  {jij  > 0}  , the  Langrangian  is  a convex  function,  and  if 
the  solution  of  VLj  *»  0,  where  V denotes  the  gradient  operator  [ZAN  69], 
is  such  that  n^  > 0,  then  it  must  be  optimal. 


11  1 ■' 11  1 ■ 1 ■■■  — !- 


VLj  = 0 *=> 


3L 


^l^m’  V = ° ^ (V 


By  equating  the  partial  derivatives  to  zero: 


ot,  -1 


al^nl(lm’  12^  = A ¥ (V  •*  i2) 


This  implies  that  all  nj's  must  be  equal.  Hence,  from  the  size 
constraint. 


n j (ijn»  1 n 


N 


1 


^ ***  ^ i •••»  i,) 

im  i, 

m 3 


By  replacing  n^  in  the  objective  function, 

,1-a,  m 

n2Cim,  ...,  i3) 

L3 


a 


N 12-  •••  2-  n2Cim,  ...,  i3)  + £ M*.  n’  a*  6 

Llm  x3  J j=2  “ ~ ~ 


B) 


ot. 


Since  B2  * N and  D2  = 1,  the  above  expressions  satisfy  Lemma  C.l. 


(ii)  General  Case 

Assuming  that  Equations  (C.l)  and  (C.2)  are  true  for 

A = 1,  2,  ...,  k , with  k < m,  let  us  prove  that  they  are  still  true 
for  l = k + 1. 

According  to  the  step-by-step  strategy,  we  propose  to  solve 
for  step  k + 1.  Let  all  n..(im,  ...,  i ) be  fixed  ¥ j - k ♦ 2,  ..., 


m 


and  let  us  solve  for 


min:  G(m,  n,  a,  8)  as  given  in  Eq.  (C.2) 


over:  {nk+1(im,  ik42)}  - nk+1  > 0. 


Notice  that  the  term,  £ G.  (m,  n,  a,  8)  in  Eq.  (C.2)  is  a constant; 

j*k*2  * ~ * * 


I 
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therefore,  it  will  be  discarded  in  the  optimization  step 


G(a,  n,  a,  $)  is  convex,  with  respect  to  n^j,  in  the  region 

2k+l  51  0t  Xt  is  sufficient  to  prove  that  Y,  the  first  term  of  Eq.  (C.2) 

is  a convex  function.  (In  what  follows,  n,(i  , i i,  ,) 

k m m-1*  ’ k+1' 

occasionally  will  be  denoted  by  n^(»).) 


which  we  define  as 


Consequently,  the  Hessian  matrix  corresponding  to  Y is 


Hy  is  equal  to  9 Y (defined  above)  times  a matrix  of  ones' s 
prove  that  Hy  is  positive  definite  in  the  region  n^+^  > 0. 


where  n. 


Therefore,  n. 


This  proves  that  Y is  a convex  function  in  the  region  of 
interest;  moreover,  since  the  second  term  of  Eq.  (C.2)  is  a sum  of 
convex  functions,  then  G(m,  n,  a,  B)  is  a convex  function. 


As  a consequence  of  Claim  C.2,  if  the  solution,  n. 


is  such  that  n.  . > 0,  then  it  must  be  the  optimal  solution 


Hence 


Notice  that  the  left-hand  side  of  this  equation  is  the  summation  of  all 


i.  ,) ; hence,  it  will  be  tho  same  for  all  (i 


Consequently,  all  nk+1's  are  equal.  Let  nk+1(in,  ....  iw,)  = n 


k+2'  "k*l. 


From  the  above  equation 


ViC  i'w'  1 ■ n («t  - 1) 

1=1  k+1 


k 

n (a  -1) 
i=l  1 


Vi  S—  .£  \.2f)  1>k*1 

L Xm  *k+3 


Grouping  the  terms  in  nk+1  to  the  left-hand  side  leads  to  nk+1  with 
the  following  exponent  (to  be  denoted:  exp  [ ] ). 


«P  (Vj!  * “k.l 


"(ai  - 11  VA.i ♦ n 

+ = 1=1 


Using  Eq.  (6.2) 


CXP  ("v.i) 


Also,  the  exponent  of  the  multiple  summation  is  equal  to 

k k 

\.i  * n (<h  - 1)  noj 

1=1  i=l 

Vi  " 

This  is  due  to  the  fact  that: 


°k.i  * n <«i 

i=l 


i)  - n«i 

i=l  1 


The  proof  of  Eq.  (C.3)  results  from  Eq.  (6.13)  as  follows: 


(C.3) 
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Finally 


Extracting  n 


would  give  Eq.  (C. 1)  for  l - k + 1.  Notice  that 


5k+i  > °-  Now»  replacing  nk+1(im 
function,  Eq.  (C.2),  we  arrive  at 


in  the  objective 


Replacing  nfc+1  above  with  its  expression  given  in  Eq.  (C.l),  leads 


As  for  Y,  we  have 


l ' U\  »; 
■ V * -*E^~  A-  ^ 


Y * 1 


k+1 

k+2 


By  performing  the  same  manipulations  as  for  X, 

k 

n («,  - 1) 

Y 


i-1  1 


k+2 

Consequently,  X = Y;  this  property  is  the  key  to  the  proof.  It  says 
that  at  each  optimization  step  we  get  one  less  term  in  the  objective 
function. 

To  prove  that  Eq.  (C.2)  is  satisfied  for  £ = k + 1,  there 
remains  to  be  shown  that  the  coefficient  of 

nk+2(V  ik+3)]X 

is  equal  to  Let  U be  that  coefficient,  then 


U = 


n Ccx  -1) 

k i=l  1 

n «*i  - 1)  B y d 

i=l  Bk+1  1 k 2 


^-“k+l^k+P 


“k+l  °k+l 


R + o°t^+1 

Bk+1  Bk+1 


n (a.  - d 

i=l  1 Bk+1 


^ “k+l^k+l^ 


“k+l  Dk+1 


where 


D n (a.  - 1) 


k+2 


k+2 
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the  term  in  the  second  bracket  becomes 


After  substitution 


as  given  in 


of  Lemma  C. 1 


Lemma  C.l  inplicity  assumed  that  k £ m - 1.  Let  us  show  that 
Eqs.  (C.l)  and  (C.2)  are  still  true  for  l * k ■ m.  Observe  that  for 


Furthermore,  by  convention,  we  set  the  value  of  the  above  sum 


to  be  1 when  £.  = m.  Consequently,  Eqs.  (C.l)  and  (C.2)  become  mean 


ingful  for  i = m.  There  remains  to  be  shown  that  these  equations 
effectively  hold  true  at  that  value  of  l = m. 


The  optimization  will  now  be  performed  over  the  single 

variable,  nm«  G,  as  given  in  Eq.  (C.4),  is  convex  in  the  region 

{nm  > 0}  . Again,  if  the  solution  n of  VG  = 0 is  such  that  n > 0 
m mm 

then  it  must  be  the  optimal  solution. 


Solving  for  n 


Tliis  result  satisfies  Eq.  (C.  1)  for  A = k = m,  where  the  summation  is 


set  ot  one 


Also  we  can  easily  check  that  the  objective  function  becomes 


since  it  is  the  last  step  in  the  optimization.  The  above  objective 
is  actually  the  minimum  computational  cost.  It  also  checks  with  the 
expression  in  Eq.  (C. 2)  where  the  summations  are  set  to  one. 


End  of  Proof  of  Propos i t ion  6.1 


From  Lemma  C. 1 and  the  extension  of  the  expresssion  in  Eq 


nA(V  14+1) 


V (i 


m’ 


W 


¥ l * 1, 


Thus,  the  multiple  summation  in  Eq.  (C.l)  becomes, 

nl*2  ■ 

*•*  .22  njUl(im 14+2)  = nmn m-1  ”*  nt*l  = ,JJ4l  "i 

The  substitution  of  the  above  summations  in  Eq.  (C.l)  results  in 
Eq.  (6.12).  Also,  Eq.  (C.5),  giving  minimus  G,  is  exactly  what  we 
have  to  prove,  i.e.,  Eq.  (6.15). 


C. 2 Application  to  Other  Special  Cases 

This  section  deals  with  the  following  two  cases: 

(i)  Uniform  design  strategy,  variable  gate  assignment,  i.e., 
o^»a  ¥ k ■ 1,  . . . , » 

B^'s  variables  (C.6) 


(ii)  Proportional  assignment  of  gates. 

In  (i),  explicit  expressions  for  the  optimal  degree  vector  and 
computational  cost,  given  m,  have  been  derived.  Some  partial  results 
related  to  the  global  optimality  have  been  found  when  is  of  the  form, 

\ - Yk'2,  k 1 2. 

f.t. 

With  respect  to  (ii),  the  number  of  k level  gates  to  be 
selected  is  proportional  to  the  number  of  k-1  level  gates  from  which 
they  are  selected.  The  corresponding  solution  was  found  to  be  of  no 
practical  interest.  The  solution  is 
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n = 0,  n = + " 
l m 

m 

n»,  ....  n , any  value  different  from  0 or  ",  s.t.  TT  n-  = N. 

£ m-i  i 1 1 

C.2.1  Uniform  Design  Strategy,  Variable  Gate  Assignment 

The  purpose  of  this  section  is  to  model  design  requirements 
whereby  different  reliability  constraints  are  imposed  on  the  design  of 
the  layer  subnets  depending  on  their  levels  in  the  hierarchy.  However, 
the  same  design  procedure  is  applied  at  all  levels. 

C.2.1.1  Explicit  Expressions 
Corollary  C.l 

Under  the  conditions  of  Proposition  6.1,  and  assuming  that  all 
the  cl’s  are  equal,  the  optimal  solution  is  such  that 


_ a f/q  - l\m  M 
nk  “ q - 1 ( q ) 


(q-l)1"^-1 
^m+1 


m m+k-i . , ^ i-k-1  k-1  „ k-i  .m-k+i-ln 

n (6.)“  (“-w  n (B.f  (a-1) 

i=k+l  1 i=l  1 

„qDm 


D 


m+1 


k 1,  2,  ...,  m 

(C.  7) 

With  this  optimal  assignment,  the  minimum  computational  cost  is 


G = D 


m+1 


(m  n 

n b“ 

i*2  1 


1"i(q-l)1~1' 


m m-1  _ D 


m+1 


(C.8) 
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From  Corollary  6.1  (Eqs.  (6.21)  and  (6.22)  we  know  that  Eqs 


(C.7)  and  (C.8)  are  true  with  regard  to  the  terms  containing  a and  N 
The  term  containing  8 in  Eq.  (C.7)  can  be  directly  obtained  from  Eq. 


(6.19).  The  rest  of  the  proof  of  Eq.  (C.8)  proceeds  by  induction 


Limiting  behavior  when  a “ 


From  Eqs.  (6.23)  and  (C.7) 


limit  n 


(C.9)  implies  that  at  the  limit  all  layer  subnets 


Since  g. 


C.2.1.2  A geometrically  monotonic  gate  assignment  strate 


In  order  to  reach  any  conclusion  with  regard  to  global 
optimality,  it  is  necessary  to  specify  8^  in  terms  of  k.  Let  8^  be  a 
geometrically  monotonic  function  of  k,  i.e.. 


Y = 1 corresponds  to  the  uniform  gate  assignment  strategy  studied  in 


Section  6.5.  With  the  above  gate  assignment  strategy,  Eqs.  (C.7)  and 


Re ma rks 


the  degree  vector  becomes 


n,  = — ° . By01"1 


n 


1 a - 1 

a ..a 


(C. 14) 


k a - 1 


k = 2,  . . . t 


Eq.  (C. 14)  is  quite  similar  to  Eq.  (6.30),  but  as  we  will  see  later, 
mj  is  not  the  optimal  number  of  levels  (except  when  Y » 1). 

(n)  In  order  to  satisfy  the  gate  constraint,  Eq.  (6.6),  the 
degree  vector,  must  be  such  that 


nl  > 6 


k ■ 2 , . . . , m 


(C. 15) 
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The  above  condition  will  always  be  satisfied  if  m nij  and  y > 1. 
Other  situations  *ay  be  checked  numerical ly. 


,1/a 


Global  Optimality 

Under  the  conditions  Ny/8  > 1 and  y > ( - ~-1  ) , the  optimal 

number  of  levels,  m„,  is  the  unique  positive  root  of  the  equation: 

B[ln  inbr] ln  = (ln  tM1"  ^rnr)  + “f1  - ]ln  Y 

(C. 16) 

The  corresponding  optimal  degree  is 


- (—) 
a-1  \ a / 


n,  = 8 


.-1 


a-1 


Y e 


ln  y 

In (a/ (a-1)) 


, m*-k  , 

I a-1  \ * ln  y 

a a " \ a / 


(C.  17) 


k a-1 


Y e 


In (a/ (a-1)) 
k 2 , « . . , m* 


Notice  that  the  expression  of  n^  is  independent  of  N. 


Proof- 


Differentiating  Eq.  (C.12)  with  respect  to  m,  we  arrive  at 


dG  _ (a  - 

d* = n 


[“”(*  ln  Br'rT  * ln  Tr)1"  iTTT  - “D».l  ln 

m+1  L J 


The  roots  and  the  sign  of  the  above  equation  can  be  determined  from  the 
study  of  Eq.  (C. 16).  Fig.  C.l  shows  a straight  line  for  the  left-hand 
side  expression  of  Eq.  (C.16)  and  a monotonically  increasing  (y  > 1) 
or  decreasing  (y  < 1)  curve  for  the  right-hand  side  of  Eq.  (C.16) 


384 


It  also  shows  a unique  intersection  point  m*.  From  the  relative 
positions  of  those  curves,  we  conclude  that  m*  leads  to  a minimum 


LEFT  HANO  SIDE  OF  EQ.  (C.16) 
RIGHT  HANO  SIDE  OF  EQ.  (C.16) 
(Lg  N7//?)  ( Log  a/  ( ;»  - 1)) 

Yc  + a Log  y 


Figur«  C.l.  Comparative  Behavior  of  the  Two  Side*  of  Eq.  (C.16) 


C.2.2  Proportional  Gate  Assignment 


In  certain  situations,  it  is  important  to  make  the  number  of 
gates  selected  at  any  level  sensitive  (e.g.,  proportional)  to  the 
number  of  nodes  of  the  set  from  which  they  are  selected.  Let  y.  (y  < 1) 
be  the  fraction  of  the  level  gates  to  be  selected  as  k«Ts*  level 


From  the  above  definitions,  the 


I 


degree  and  the  size  vectors  are  such  that 


glfV  •••  12)  = 


gk(V  k+P 


nl(V  12) 

nkCV  ••••  W (c-18^ 

X Yk-lgk-l(V  V 


k ~ 2,  • • • p in 


thus. 


gk(V 


n (i  i ) n2(lm’  i3) 

V ■ W _ . k-l  k 


i^(  j?j  Yj)nl(im \ 


(C. 19) 


The  general  expression  of  the  computational  cost  of  all  kth  level  layer 


subnets  is 


\n  Vl(V  ik*2)  a. 


V"*  n,  a,  Y)  = £ •••  £ t8k(im,  ik+,l  k (C.20) 


i =1  i,  . = 1 
m k+1 


Also,  Eq.  (6.9)  still  holds  true. 

Substituting  Eq.  (C. 19)  into  Eq.  (C.20)  for  k = m,  we  find 

n2(V  *•*»  i3) 

r n , n 

_2J  _ / m-l  \ *1  m 

Gm  = Xj  ”*  Xjl  £ Yj)  nl(im*  i2) 

L m 2 1 J~1 


Consequently,  if  n satisfies  the  size  constraint,  Eq.  (2.1) 


o[i5>r 
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Therefore,  if  for  a particular  feasible  solution  n,  G is  equal  to  its 
lower  bound  given  above,  then  that  vector  n is  optimal. 


Proposition  C. 2 


Under  the  proportional  gate  assignment  and  for  > 1, 

(k  = 1,  ....  m)  and  a fixed  m,  an  optimal  solution  of  Problem  6.11 


is  n*. 


n*  = 0 


nw  = + oo 

m 


n2»  n*,  ...,  n*  j > 0,  / “ s.t.  n?  = N 

i=l  1 


With  this  assignment 


(m-1  ia  a 

n yij  m n m 


Proof: 


Let  n be  a feasible  vector,  and  such  that 


(C. 22) 


nk(V  ik+l)  =nk  Vk-1,  ....  m-1 


JT  n = N 
k=l  k 


(C. 23) 


From  Eqs . (C.19),  (C.20)  and  (C.23),  we  jet 
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o.-l  m-1  j k-1  va.  / m .1-ol  /«-l  .a  a 
G(»,  n,  a,  y)  = N(n  ) 1 ♦ £ (N  II  Y4 1 ( IT  n.)  ♦(  IT  Y± ) m N m 

" ~ ~ 1 k"'  i=i  V 'i*k*i  v H=i  v 


For  n equals  n*,  (Eq.  (C. 22)) 


/ ■ X1 
limit  II  n.  ) 

n -**»  \i=k+l  V 


k = 2 j •#*}  m * 1 


/m~l  <* 

C(m,  n*,  a,  y)  «(  JI  Y^  N " 


Hence , 


Then,  because  of  Eq.  (C.21),  n*  is  optimal. 

For  a realistic  network,  this  optimal  solution  is  meaningless, 
and  it  is  necessary  either  to  keep  the  integer  constraint  on  n or  to 
introduce  other  constraints. 


